-- Logs begin at Mon 2022-05-23 11:58:30 UTC, end at Mon 2022-05-23 13:30:26 UTC. -- May 23 12:50:31.853530 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 23 12:50:31.862199 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 12:50:31 2022] *** May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: nodename: dv-cinder-master-3-iscsi May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: machine: x86_64 May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: clock source: unix May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: pcre jit disabled May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: detected number of CPU cores: 4 May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: current working directory: / May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: detected binary path: /usr/bin/uwsgi-core May 23 12:50:31.864300 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: your processes number limit is 47680 May 23 12:50:31.865097 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: your memory page size is 4096 bytes May 23 12:50:31.865097 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: detected max file descriptor number: 2048 May 23 12:50:31.865097 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: lock engine: pthread robust mutexes May 23 12:50:31.865097 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: thunder lock: enabled May 23 12:50:31.865097 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 23 12:50:31.865097 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 12:50:31.909526 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: Python main interpreter initialized at 0x5579ba50a9c0 May 23 12:50:31.909526 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: python threads support enabled May 23 12:50:31.909526 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: your server socket listen backlog is limited to 100 connections May 23 12:50:31.909526 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: your mercy for graceful operations on workers is 90 seconds May 23 12:50:31.909847 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: mapped 403077 bytes (393 KB) for 2 cores May 23 12:50:31.910015 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: *** Operational MODE: preforking *** May 23 12:50:31.910274 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: *** uWSGI is running in multiple interpreter mode *** May 23 12:50:31.910274 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: spawned uWSGI master process (pid: 115195) May 23 12:50:31.910460 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 23 12:50:31.912119 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: spawned uWSGI worker 1 (pid: 115196, cores: 1) May 23 12:50:31.912119 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: spawned uWSGI worker 2 (pid: 115197, cores: 1) May 23 12:50:31.912119 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 12:50:33.081222 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 12:50:33.082879 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 12:50:33.804841 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:50:33.804841 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: from cryptography.utils import int_from_bytes May 23 12:50:33.805093 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:50:33.805093 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: from cryptography.utils import int_from_bytes May 23 12:50:33.805214 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:50:33.805214 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: from cryptography.utils import int_from_bytes May 23 12:50:33.805329 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:50:33.805329 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: from cryptography.utils import int_from_bytes May 23 12:50:36.391289 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 12:50:36.391874 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 12:50:36.391965 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 12:50:36.391965 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 12:50:36.392376 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 12:50:36.392376 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 12:50:36.392514 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 12:50:36.392651 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 12:50:36.392798 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 12:50:36.393004 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 12:50:36.393096 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.393337 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.393426 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.393657 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.393744 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.394007 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.394110 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.400977 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401077 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401149 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401216 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401291 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401367 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401435 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401501 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401566 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401637 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401719 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401788 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401862 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.401964 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402031 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402096 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402179 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402247 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402320 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402387 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402453 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402524 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402598 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402670 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402744 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402824 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402892 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.402971 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403290 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403362 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403433 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403499 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403589 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403676 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403750 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403842 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403910 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.403975 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404052 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404120 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404187 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404262 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404338 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404410 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404484 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404551 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404616 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404689 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404762 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404828 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404923 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.404999 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405073 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405153 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405227 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405301 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405374 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405446 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405521 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405588 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405652 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405717 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405795 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405795 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.405970 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406047 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406113 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406203 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406277 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406351 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406418 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406483 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.406548 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407430 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407534 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407621 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407702 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407781 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407859 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.407951 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408032 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408116 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408246 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408337 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408430 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408517 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408603 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408730 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.408890 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409029 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409200 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409327 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409415 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409607 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409705 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.409937 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.410070 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.410260 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.410395 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.410709 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.410816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411034 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411142 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411292 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411403 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411536 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411629 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411847 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.411945 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.412163 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.412252 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.412469 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.412578 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.412771 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.412859 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.413077 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.413162 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.413379 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.413473 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.413683 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.413768 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414010 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414154 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414315 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414403 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414621 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414715 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.414937 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.415032 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.415247 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.415339 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.415565 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.415669 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.415896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416003 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416212 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416300 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416601 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.30 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416687 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.30 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416849 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.30 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.416934 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.30 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.417150 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.417235 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.417483 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.417579 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.417796 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.417940 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.418127 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.418217 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.418440 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.418533 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.418756 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.418850 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419059 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419149 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419360 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419446 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419661 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419756 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.419977 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420135 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420295 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420382 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420595 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420690 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420843 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.420928 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.421138 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.421224 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.421440 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.421535 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.421753 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.421848 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422079 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422168 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422383 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422470 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422684 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422770 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.422985 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423071 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423287 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423375 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423589 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423675 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423891 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.423984 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.424193 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.424278 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.424495 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.424580 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.424795 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.424882 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.425098 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.425192 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.425400 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.425486 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426035 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426126 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426225 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426314 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426395 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426474 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426672 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426768 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.426977 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427064 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427283 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427383 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427588 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427689 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427902 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.427996 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.428205 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.428291 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.428508 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.428595 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.428826 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.428919 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.429125 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.429211 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.429426 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.429527 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.429738 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.429823 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430066 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430154 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430368 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430471 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430683 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430770 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.430977 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431061 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431279 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431373 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431586 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431680 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431888 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.431981 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.432187 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.432282 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.432486 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.432578 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.432790 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.432885 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.433094 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.433194 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.433396 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.433483 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.433695 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.433784 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434021 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434111 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434320 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434406 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434634 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434720 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.434954 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435043 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435258 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435351 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435560 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435648 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435862 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.435961 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.436176 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.436275 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:50:36.436505 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.436590 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.436829 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.436915 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.437164 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.437251 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.437472 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.437576 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.437797 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.437915 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.438145 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.438230 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.438459 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.438559 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.438766 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.438850 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.439072 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.439168 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.439394 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.439496 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.439700 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.439786 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440004 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440097 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440310 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440397 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440621 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440705 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.440933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.441045 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442025 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442145 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442145 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442315 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442397 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442475 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442711 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.442813 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443019 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443105 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443329 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443414 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443652 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443748 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.443971 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.444057 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.444276 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.444408 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.444613 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.444702 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.444951 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.445039 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.445260 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.445347 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.445580 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.445668 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.445913 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446015 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446227 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446331 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446534 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446623 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446840 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.446927 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.447159 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.447259 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.447487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.447580 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.447793 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.447895 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.448106 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.448203 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.448416 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.448503 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.448722 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.448813 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449028 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449126 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449332 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449426 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449640 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449726 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.449995 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.450084 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.450306 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.450405 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.450619 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.450708 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.450930 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451044 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451248 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451347 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451555 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451658 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451860 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.451947 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.452167 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.452252 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.452463 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.452555 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.452764 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.452861 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.453064 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.453150 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.453366 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.453451 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.453666 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.453769 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454003 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454101 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454324 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454425 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454625 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454714 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.454954 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455042 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455257 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455352 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455564 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455651 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455871 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.455958 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.456178 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.456271 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.456476 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.456562 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.457366 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.457859 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.457859 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458066 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458066 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458214 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458293 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458378 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458589 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458679 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458891 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.458986 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.459208 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.459295 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.459510 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.459600 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.459810 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.459897 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.460117 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.460203 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.460423 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.460512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.460732 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.460822 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461037 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461125 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461345 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461432 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461661 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461754 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.461992 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.462095 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.462302 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.462390 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.462630 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.462719 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.462958 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463056 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463259 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463346 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463580 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463666 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463900 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.463995 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.464210 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.464297 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.464507 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.464641 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.464814 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.464909 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.465113 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.465200 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.465414 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.465512 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.465711 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.465798 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466036 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466126 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466337 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466425 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466637 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466725 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.466936 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467039 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467233 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467329 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467530 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467617 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467844 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.467936 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.468150 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.468245 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.468451 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.468545 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.468749 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.468834 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.469048 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.469145 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.469388 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.469475 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.469687 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.469774 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.470044 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.470150 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.470375 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.470462 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.470690 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.470790 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471030 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471133 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471318 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471403 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471629 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471714 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.471933 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.472031 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.472241 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.472395 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473003 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473110 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473208 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473291 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473370 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473449 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473580 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473676 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.473917 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474021 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474230 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474319 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474550 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474639 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474859 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.474950 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.475177 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.475279 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.475480 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.475568 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.475793 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.475882 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.476128 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.476215 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.476454 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.476539 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.476769 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.476857 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.477086 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.477173 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.477394 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.477488 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.477703 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.477789 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478037 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478127 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478346 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478444 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478741 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.478960 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479046 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479276 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479371 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479583 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479708 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479897 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.479982 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.480203 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.480302 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.480518 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.480603 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.480823 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.480908 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.481134 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.481228 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.481435 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.481521 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.481742 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.481834 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.482086 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.482175 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.482413 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.482498 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.482720 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.482808 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483024 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483119 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483340 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483434 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483642 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483728 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.483945 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484033 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484251 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484349 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484547 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484848 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.484935 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.485152 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.485238 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.485457 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.485551 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.485763 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.485856 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.486089 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.486179 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.486389 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.486488 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.486700 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.486795 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487006 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487093 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487307 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487403 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487606 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487692 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.487932 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.488084 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.488632 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.488728 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.488809 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.488888 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.488967 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489167 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489255 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489470 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489557 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489774 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.489861 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.490114 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.490202 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.490421 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.490515 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.490717 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.490803 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491018 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491106 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491404 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491651 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491741 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.491956 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492045 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492258 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492352 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492567 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492671 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492884 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.492970 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.493187 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.493272 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.493496 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.493593 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.493798 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.493903 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.494123 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.494217 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.494439 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.494535 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.494747 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.494836 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495049 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495140 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495353 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495451 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495660 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495747 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.495962 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496048 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496269 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496367 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496585 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496679 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496889 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.496975 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.497193 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.497279 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.497497 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.497590 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.497797 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.497974 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.498124 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.498213 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.498437 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.498522 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.498997 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.499085 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.499325 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.499418 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.499635 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.499720 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.499949 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.500044 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.500262 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.500356 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.500575 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.500660 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.500903 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501003 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501227 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501312 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501568 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501664 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501871 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.501988 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.502197 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.502284 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.502497 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.502584 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.502801 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.502888 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.503105 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.503200 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.503406 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.503499 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.504229 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.504637 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.504741 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.504833 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.504915 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.504994 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.505113 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.505264 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.505415 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.505567 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.505716 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.505867 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506042 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506187 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506355 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506497 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506656 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506796 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.506965 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.507102 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.507266 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.507400 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.507586 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.507719 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.507894 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508025 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508205 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508294 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508507 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508601 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508808 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.508895 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.509148 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.509240 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.509450 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.509545 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.509784 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.509904 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.510136 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.510224 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.510468 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.510555 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.510768 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.510867 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511067 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511196 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511386 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511473 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511684 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511778 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.511982 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512068 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512278 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512364 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512581 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512741 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512881 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.512967 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.513179 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.513274 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.513483 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.513570 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.513787 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.513925 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.514114 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.514200 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.514419 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.514506 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.514717 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.514802 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515020 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515105 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515315 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515401 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515623 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515708 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.515922 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516007 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516220 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516306 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516513 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516598 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.516911 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.517118 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.517203 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.517413 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.517498 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.517718 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.517819 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518038 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518183 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518348 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518437 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518645 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518742 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.518943 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.519029 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.519821 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520373 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520373 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520532 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520615 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520694 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520773 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.520858 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521072 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521301 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521379 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521446 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521612 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521701 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.521936 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522029 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522247 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522341 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522546 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522635 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522846 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.522935 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.523155 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.523240 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.523472 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.523560 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.523782 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.523869 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524078 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524176 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524382 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524476 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524679 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524765 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.524977 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.525065 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.525276 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.525370 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.525577 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.525663 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.525889 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526037 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526191 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526285 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526573 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526781 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.526874 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527078 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527177 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527393 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527486 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527698 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527783 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.527994 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.528332 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.528550 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.528637 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.528848 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.528934 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.529183 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.529270 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.529482 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.529568 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.529836 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.529941 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.530162 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.530250 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.530470 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.530557 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.530785 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.530872 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.531093 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.531181 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.531401 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.531487 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.531708 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.531793 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532008 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532096 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532320 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532406 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532625 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532719 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.532933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.533028 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.533236 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.533324 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.533533 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.533631 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.533848 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534010 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534183 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534270 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534592 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534836 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.534980 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.535475 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.535562 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.535649 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.535739 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.535821 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.535899 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536053 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536141 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536363 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536451 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536672 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536759 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.536981 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.537066 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.537288 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.537376 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.537595 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.537684 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.537919 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538008 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538229 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538316 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538538 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538625 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538845 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.538932 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.539147 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.539236 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.539454 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.539548 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.539758 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.539855 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540059 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540145 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540376 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540462 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540684 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540784 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.540982 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.541068 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.541287 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.541374 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.541587 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.541755 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.541922 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.542011 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.542214 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.542309 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.542633 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.542729 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.542944 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543031 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543251 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543337 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543548 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543635 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543855 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.543949 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.544160 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.544255 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.544470 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.544562 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.544778 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.544873 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.545080 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.545168 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.545489 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.545577 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.545795 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.545909 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.546124 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.546214 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.546432 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.546518 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.546727 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.546813 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547026 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547129 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547331 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547419 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547651 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547745 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.547949 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548044 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548246 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548544 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548677 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548858 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.548954 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.549156 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.549244 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.549471 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.549558 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.549922 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.550032 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.550208 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.550295 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.550544 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.550715 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551147 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551242 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551325 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551411 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551539 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551677 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551860 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.551956 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.552182 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.552278 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.552504 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.552593 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.552807 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.552917 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.553117 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.553207 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.553420 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.553506 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.553742 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.553829 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.554095 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.554193 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.554407 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.554496 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.554721 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.554811 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555025 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555114 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555343 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555439 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555661 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555763 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.555969 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.556065 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.556287 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.556374 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.556692 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.556787 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557010 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557103 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557312 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557398 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557628 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557725 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.557971 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558061 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558272 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558358 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558573 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558674 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558879 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.558965 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.559175 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.559261 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.559475 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.559563 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.559792 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.559880 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.560091 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.560186 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.560398 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.560491 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.560702 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.560787 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561005 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561091 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561303 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561391 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561600 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561686 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.561933 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562022 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562234 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562323 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562552 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562645 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562857 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.562952 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.563174 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.563267 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.563487 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.563573 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.563787 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.563882 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.564099 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.564185 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.564402 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.564491 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.564703 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.564804 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565004 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565090 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565301 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565393 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565638 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565779 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.565958 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.566552 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.566826 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567210 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567308 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567391 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567509 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567599 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.567942 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.568146 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.568234 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.568462 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.568556 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.568774 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.568870 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.569088 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.569185 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.569408 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.569502 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.569705 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.569798 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570025 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570116 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570334 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570421 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570631 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570726 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.570931 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571017 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571227 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571313 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571532 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571626 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571828 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.571922 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.572123 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.572212 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.572419 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.572515 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.572732 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.572827 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573028 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573114 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573343 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573442 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573733 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.573977 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574068 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574278 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574378 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574576 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574662 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574911 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.574997 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.575217 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.575307 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.575533 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.575623 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.575843 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.575942 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.576159 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.576254 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.576476 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.576563 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.576777 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.576865 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.577093 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.577183 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.577404 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.577494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.577713 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.577809 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578045 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578135 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578354 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578453 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578662 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578763 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.578969 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579063 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579275 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579369 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579577 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579677 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579887 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.579973 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.580192 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.580277 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.580503 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.580600 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.581014 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.581101 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.581337 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.581423 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.581647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.582730 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.582730 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.582895 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.582895 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583039 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583184 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583282 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583498 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583586 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583811 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.583906 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.584121 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.584211 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.584431 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.584526 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.584739 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.584836 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.585062 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.585148 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.585380 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.585467 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.585699 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.585794 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586054 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586146 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586367 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586456 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586677 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586775 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.586987 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.587084 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.587294 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.587390 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.587642 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.587734 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.587952 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588045 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588255 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588353 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588565 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588652 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588874 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.588959 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.589184 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.589278 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.589491 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.589584 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.589799 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.589901 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.590132 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.590229 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.590440 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.590526 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.590744 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.590873 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591050 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591146 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591356 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591442 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591665 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591753 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.591985 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.592072 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.592311 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.592405 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.592627 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.592715 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.592935 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.593025 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.593245 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.593340 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.593560 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.593647 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.594099 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.594190 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.594413 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.594501 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.594741 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.594840 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.595061 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.595149 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.595380 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.595482 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.595691 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.595788 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596004 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596131 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596325 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596413 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596626 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596759 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.596925 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.597012 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.597232 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.597338 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.597831 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.597956 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.597956 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598116 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598204 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598360 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598493 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598674 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598793 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.598984 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.599108 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.599305 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.599401 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.599615 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.599928 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.600149 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.600271 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.600500 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.600597 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.600821 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.600917 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.601150 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.601237 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.601477 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.601574 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.601789 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.601895 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.602132 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.602220 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.602434 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.602521 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.602737 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.602824 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603047 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603140 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603359 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603449 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603657 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603745 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.603955 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604043 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604258 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604346 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604561 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604654 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604862 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.604951 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.605163 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.605258 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.605460 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.605546 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.605763 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.605863 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606090 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606187 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606389 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606478 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606688 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606786 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.606991 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607081 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607286 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607371 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607583 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607679 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607883 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.607971 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.608180 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.608268 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.608481 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.608581 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.608777 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.608863 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609080 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609168 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609379 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609475 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609676 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609785 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.609995 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.610099 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.610299 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.610387 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.610608 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.610695 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.610941 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611027 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611244 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611342 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611540 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611629 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611842 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.611928 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.612145 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.612232 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.612451 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.612545 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.612754 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.612849 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.613093 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.613553 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.613639 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.613719 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.613799 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.613994 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.614261 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.614350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.614591 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.614834 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.614923 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.615171 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.615270 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.615493 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.615580 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.615824 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.615910 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.616126 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.616214 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.616431 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.616519 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.616740 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.616835 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617045 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617131 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617352 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617441 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617660 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617756 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.617988 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618094 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618296 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618385 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618601 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618698 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618906 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.618995 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.619209 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.619297 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.619515 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.619607 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.619820 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.619906 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.620123 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.620209 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.620427 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.620516 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.620735 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.620828 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.621044 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.621143 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.621357 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.621447 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.621669 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.621755 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622005 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622107 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622320 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622415 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622627 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622722 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.622932 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623028 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623239 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623325 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623542 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623635 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623846 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.623940 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.624153 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.624240 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.624460 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.624545 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.624760 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.624857 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.625079 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.625165 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.625383 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.625471 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.625760 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.625856 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.626106 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.626196 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.626413 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.626500 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.626731 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.626821 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627038 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627123 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627342 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627442 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627655 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627751 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.627960 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.628048 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.628276 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.628376 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.629105 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.629663 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.629663 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.629815 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.629917 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630016 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630100 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630179 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630335 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630504 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630635 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630812 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.630940 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.631116 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.631254 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.631422 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.631557 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.631731 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.631854 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632050 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632139 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632357 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632453 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632657 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632745 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.632957 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633045 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633264 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633361 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633566 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633654 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633868 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.633986 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.634185 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.634280 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.634484 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.634572 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.634787 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.634878 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635088 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635184 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635388 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635475 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635688 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635783 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.635986 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636076 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636285 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636431 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636583 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636673 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636885 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.636974 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.637183 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.637270 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.637479 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.637564 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.637777 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.637864 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.638106 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.638195 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.638407 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.638508 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.638737 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.638826 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639040 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639127 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639377 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639475 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639663 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639751 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.639978 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640071 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640293 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640456 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640598 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640686 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640900 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.640987 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.641222 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.641355 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.641527 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.641615 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.641831 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.641949 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.642158 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.642253 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.642465 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.642553 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.642769 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.642869 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.643072 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.643161 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.643392 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.643481 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.643717 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.643813 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.644018 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.644108 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.644425 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.645300 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.645386 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.645477 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.645723 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.645818 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.646062 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.646152 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.646372 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.646518 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.646698 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.646787 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647006 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647094 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647311 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647413 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647622 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647711 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.647927 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648015 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648231 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648328 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648541 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648629 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648848 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.648934 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.649162 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.649258 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.649477 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.649563 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.649783 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.649891 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.650126 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.650215 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.650448 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.650536 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.650755 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.650850 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.651089 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.651197 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.651405 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.651535 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.651723 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.651818 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652032 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652121 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652339 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652434 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652639 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652735 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.652879 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115196) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 12:50:36.652993 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.653561 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.653866 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:50:36.654131 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115197) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 12:50:36.712936 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO dbcounter [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Registered counter for database nova_api May 23 12:50:36.713607 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO dbcounter [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Registered counter for database nova_api May 23 12:50:36.733794 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_db.sqlalchemy.engines [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115196) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.735327 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_db.sqlalchemy.engines [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.741128 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writer thread running {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.742372 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writer thread running {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.797519 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.context [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3c6cf08e-95e3-44d9-8446-eef7c93b66c5(cell1) {{(pid=115196) load_cells /opt/stack/nova/nova/context.py:464}} May 23 12:50:36.798593 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.context [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3c6cf08e-95e3-44d9-8446-eef7c93b66c5(cell1) {{(pid=115197) load_cells /opt/stack/nova/nova/context.py:464}} May 23 12:50:36.802276 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:50:36.803188 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_concurrency.lockutils [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115197) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:50:36.803867 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:50:36.804650 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:50:36.804795 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_concurrency.lockutils [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115197) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:50:36.805576 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_concurrency.lockutils [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115197) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:50:36.806619 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:50:36.807583 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_concurrency.lockutils [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115197) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:50:36.808583 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO dbcounter [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Registered counter for database nova_cell0 May 23 12:50:36.809558 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO dbcounter [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Registered counter for database nova_cell0 May 23 12:50:36.812544 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO dbcounter [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Registered counter for database nova_cell1 May 23 12:50:36.813481 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO dbcounter [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Registered counter for database nova_cell1 May 23 12:50:36.820438 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_db.sqlalchemy.engines [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115196) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.822124 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_db.sqlalchemy.engines [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115196) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.824284 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_db.sqlalchemy.engines [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.826343 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_db.sqlalchemy.engines [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.826597 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writer thread running {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.831733 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writer thread running {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.834488 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writer thread running {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.965073 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writer thread running {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.974966 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO dbcounter [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Registered counter for database nova_cell0 May 23 12:50:36.980351 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO dbcounter [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Registered counter for database nova_cell0 May 23 12:50:36.983666 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_db.sqlalchemy.engines [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115196) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.988639 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writer thread running {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:36.989516 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG oslo_db.sqlalchemy.engines [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:50:36.993827 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writer thread running {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:50:37.390246 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: WARNING keystonemiddleware.auth_token [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:50:37.398109 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: WARNING keystonemiddleware.auth_token [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:50:37.421998 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: WARNING keystonemiddleware.auth_token [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:50:37.429503 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: WARNING keystonemiddleware.auth_token [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:50:37.743054 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: WARNING keystonemiddleware.auth_token [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:50:37.748961 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: WARNING keystonemiddleware.auth_token [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:50:37.754742 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5579ba50a9c0 pid: 115196 (default app) May 23 12:50:37.756771 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:50:37.757956 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] 10.222.0.30 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001706 May 23 12:50:37.758369 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 788 bytes} [Mon May 23 12:50:37 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:50:37.775490 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: WARNING keystonemiddleware.auth_token [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:50:37.781676 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: WARNING keystonemiddleware.auth_token [None req-db3906a1-ad91-427a-a20e-d2a44b59b123 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:50:37.787477 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5579ba50a9c0 pid: 115197 (default app) May 23 12:51:58.535590 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:51:58.536596 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001440 May 23 12:51:58.537154 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 2/2] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:51:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:51:58.548945 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writing DB stats nova_api:SELECT=3 {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:51:58.549827 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writing DB stats nova_cell0:SELECT=3 {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:51:58.550621 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writing DB stats nova_cell1:SELECT=3 {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:51:58.551537 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:51:58.991579 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-1c7f9d57-3266-41f1-b81f-2bce65bc1627 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:51:59.022029 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-1c7f9d57-3266-41f1-b81f-2bce65bc1627 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.478467 May 23 12:51:59.022914 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 1/3] 10.222.0.30 () {60 vars in 1291 bytes} [Mon May 23 12:51:58 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:01.311102 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:01.312100 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-c92f2581-6506-4363-ad96-c773deead1c5 None None] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001405 May 23 12:52:01.312488 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 3/4] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:01.375118 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-7969b17a-d58f-4a2e-b6cd-930d96e33367 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "rxtx_factor": 1.0, "id": "c1", "os-flavor-access:is_public": true, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "name": "cirros256", "swap": 0}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:01.394079 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:52:01.394079 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: warnings.warn(msg) May 23 12:52:01.491000 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-7969b17a-d58f-4a2e-b6cd-930d96e33367 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.171601 May 23 12:52:01.491834 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 2/5] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:01 2022] POST /compute/v2.1/flavors => generated 407 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 12:52:01.506802 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_api:SELECT=3 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:01.507681 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_cell0:SELECT=3 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:01.509009 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_cell1:SELECT=3 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:01.509821 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:01.713240 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-d8cddac0-f5a8-4c1e-960f-5ecd8382a248 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:01.715690 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:52:01.715690 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: warnings.warn(msg) May 23 12:52:01.862082 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-d8cddac0-f5a8-4c1e-960f-5ecd8382a248 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.358991 May 23 12:52:01.874063 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 4/6] 10.222.0.30 () {64 vars in 1335 bytes} [Mon May 23 12:52:01 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 12:52:04.184975 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-7969b17a-d58f-4a2e-b6cd-930d96e33367 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:04.186030 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-7969b17a-d58f-4a2e-b6cd-930d96e33367 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 May 23 12:52:04.186438 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 3/7] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:04.295695 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-e294ae82-ca00-4227-af45-5654b880a2ff admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "name": "ds512M", "os-flavor-access:is_public": true, "disk": 5, "id": "d1", "vcpus": 1, "ram": 512, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:04.314616 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:52:04.314616 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: warnings.warn(msg) May 23 12:52:04.417166 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-e294ae82-ca00-4227-af45-5654b880a2ff admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.224164 May 23 12:52:04.417739 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 5/8] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:04 2022] POST /compute/v2.1/flavors => generated 404 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:04.430222 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-dea74498-faf3-4ff2-bf91-ffc25f86df7e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:04.431813 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:52:04.431813 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: warnings.warn(msg) May 23 12:52:04.524858 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-dea74498-faf3-4ff2-bf91-ffc25f86df7e admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.098314 May 23 12:52:04.525667 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 4/9] 10.222.0.30 () {64 vars in 1335 bytes} [Mon May 23 12:52:04 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:06.810082 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-e294ae82-ca00-4227-af45-5654b880a2ff admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:06.811105 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-e294ae82-ca00-4227-af45-5654b880a2ff admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001518 May 23 12:52:06.811503 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 6/10] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:06.878519 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b9e98e5d-ba0f-4086-8ff8-923b3419c63a admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds1G", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 10, "swap": 0, "vcpus": 1, "id": "d2", "rxtx_factor": 1.0, "ram": 1024}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:06.965737 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-b9e98e5d-ba0f-4086-8ff8-923b3419c63a admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.148105 May 23 12:52:06.966619 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 5/11] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:06 2022] POST /compute/v2.1/flavors => generated 404 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:06.978950 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-52ff5b3a-2cfd-41b1-b3fd-a4fe3101628b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:07.066603 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-52ff5b3a-2cfd-41b1-b3fd-a4fe3101628b admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.091403 May 23 12:52:07.067584 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 7/12] 10.222.0.30 () {64 vars in 1335 bytes} [Mon May 23 12:52:06 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 12:52:09.332335 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b9e98e5d-ba0f-4086-8ff8-923b3419c63a admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:09.333354 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-b9e98e5d-ba0f-4086-8ff8-923b3419c63a admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001469 May 23 12:52:09.333762 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 6/13] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:09 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:09.401263 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-38378ae4-7959-462c-a2d2-95b6122be5e4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "id": "d3", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 10, "ram": 2048, "name": "ds2G", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "swap": 0}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:09.473541 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-38378ae4-7959-462c-a2d2-95b6122be5e4 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.133281 May 23 12:52:09.474097 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 8/14] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:09 2022] POST /compute/v2.1/flavors => generated 404 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:09.486070 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-555dc00a-87e4-4270-9c84-68d8eff31818 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:09.581113 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-555dc00a-87e4-4270-9c84-68d8eff31818 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.098862 May 23 12:52:09.582026 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 7/15] 10.222.0.30 () {64 vars in 1335 bytes} [Mon May 23 12:52:09 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:11.848467 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-38378ae4-7959-462c-a2d2-95b6122be5e4 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:11.849459 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-38378ae4-7959-462c-a2d2-95b6122be5e4 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 May 23 12:52:11.850075 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 9/16] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:11.921308 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b0a32d95-f50c-41f1-a43a-50d2620a4747 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 20, "name": "ds4G", "vcpus": 4, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 4096, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:12.165814 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-b0a32d95-f50c-41f1-a43a-50d2620a4747 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.309864 May 23 12:52:12.177141 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 8/17] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 12:52:12.179660 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-51864da2-c07c-47e8-91b8-84000331edfa admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:12.312753 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-51864da2-c07c-47e8-91b8-84000331edfa admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.137130 May 23 12:52:12.313674 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 10/18] 10.222.0.30 () {64 vars in 1335 bytes} [Mon May 23 12:52:12 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:14.588277 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b0a32d95-f50c-41f1-a43a-50d2620a4747 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:14.589299 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-b0a32d95-f50c-41f1-a43a-50d2620a4747 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 May 23 12:52:14.589705 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 9/19] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:14.652665 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-e3475951-6164-4072-9979-c1574ccad71f admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:14.676697 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-e3475951-6164-4072-9979-c1574ccad71f admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.080331 May 23 12:52:14.678463 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 11/20] 10.222.0.30 () {60 vars in 1291 bytes} [Mon May 23 12:52:14 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:16.943125 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b0a32d95-f50c-41f1-a43a-50d2620a4747 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:16.944134 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-b0a32d95-f50c-41f1-a43a-50d2620a4747 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 May 23 12:52:16.944532 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 10/21] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:17.008287 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-cc392f6f-c4f2-4f2c-afa0-d3c067ce0f19 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "1", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.tiny", "ram": 512, "rxtx_factor": 1.0, "swap": 0, "vcpus": 1, "disk": 1}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:17.087477 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-cc392f6f-c4f2-4f2c-afa0-d3c067ce0f19 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.136934 May 23 12:52:17.088353 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 12/22] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:16 2022] POST /compute/v2.1/flavors => generated 402 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:17.102963 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-d1986e14-9eb9-4f06-8c07-07a2efca9a7e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:17.174125 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-d1986e14-9eb9-4f06-8c07-07a2efca9a7e admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.076387 May 23 12:52:17.174726 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 11/23] 10.222.0.30 () {64 vars in 1332 bytes} [Mon May 23 12:52:17 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:19.993848 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-cc392f6f-c4f2-4f2c-afa0-d3c067ce0f19 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:19.995209 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-cc392f6f-c4f2-4f2c-afa0-d3c067ce0f19 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002014 May 23 12:52:19.995762 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 13/24] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:20.058891 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-2038a60a-2b63-48b1-95b7-20c4ecfd93ad admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "id": "2", "swap": 0, "os-flavor-access:is_public": true, "disk": 20, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.small", "ram": 2048}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:20.153536 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-2038a60a-2b63-48b1-95b7-20c4ecfd93ad admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.149541 May 23 12:52:20.154139 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 12/25] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:20 2022] POST /compute/v2.1/flavors => generated 405 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 12:52:20.165934 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-55d7c488-913d-409c-b8e8-f20f69092b93 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:20.248547 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-55d7c488-913d-409c-b8e8-f20f69092b93 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.086175 May 23 12:52:20.249051 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 14/26] 10.222.0.30 () {64 vars in 1332 bytes} [Mon May 23 12:52:20 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:22.672268 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-2038a60a-2b63-48b1-95b7-20c4ecfd93ad admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:22.673675 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-2038a60a-2b63-48b1-95b7-20c4ecfd93ad admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002403 May 23 12:52:22.674308 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 13/27] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:22 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:22.735213 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-7e8d3b9a-6fb3-4e05-85e3-bc4c8e6902d7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.medium", "os-flavor-access:is_public": true, "swap": 0, "id": "3", "disk": 40, "ram": 4096, "vcpus": 2, "rxtx_factor": 1.0}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:22.826768 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-7e8d3b9a-6fb3-4e05-85e3-bc4c8e6902d7 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.146949 May 23 12:52:22.827575 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 15/28] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:22 2022] POST /compute/v2.1/flavors => generated 406 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:22.839985 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-a93037b0-b792-4f05-9a26-a3323669e326 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:23.059997 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-a93037b0-b792-4f05-9a26-a3323669e326 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.223694 May 23 12:52:23.060477 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 14/29] 10.222.0.30 () {64 vars in 1332 bytes} [Mon May 23 12:52:22 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 12:52:25.398150 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-7e8d3b9a-6fb3-4e05-85e3-bc4c8e6902d7 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:25.399088 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-7e8d3b9a-6fb3-4e05-85e3-bc4c8e6902d7 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 May 23 12:52:25.399485 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 16/30] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:25.460536 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-1865ed96-defa-40b8-a030-8ecfbd41ee47 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "id": "4", "disk": 80, "swap": 0, "ram": 8192, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "os-flavor-access:is_public": true}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:25.553333 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-1865ed96-defa-40b8-a030-8ecfbd41ee47 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.147871 May 23 12:52:25.553914 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 15/31] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:25 2022] POST /compute/v2.1/flavors => generated 405 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:25.565629 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-d2ed70da-a32b-4555-b49e-ee4e1d5b24b8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:25.648625 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-d2ed70da-a32b-4555-b49e-ee4e1d5b24b8 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.086492 May 23 12:52:25.649231 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 17/32] 10.222.0.30 () {64 vars in 1332 bytes} [Mon May 23 12:52:25 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 12:52:27.945416 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-1865ed96-defa-40b8-a030-8ecfbd41ee47 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:27.946471 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-1865ed96-defa-40b8-a030-8ecfbd41ee47 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001498 May 23 12:52:27.946902 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 16/33] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:28.009242 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-307353d6-a0f6-426e-88d9-951b6a811032 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "m1.xlarge", "vcpus": 8, "swap": 0, "rxtx_factor": 1.0, "id": "5", "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 16384}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:28.073631 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-307353d6-a0f6-426e-88d9-951b6a811032 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.120841 May 23 12:52:28.074751 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 18/34] 10.222.0.30 () {64 vars in 1282 bytes} [Mon May 23 12:52:27 2022] POST /compute/v2.1/flavors => generated 408 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:28.087061 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-4fe01a51-2b88-4096-b87f-dac03242c901 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:28.173763 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-4fe01a51-2b88-4096-b87f-dac03242c901 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090497 May 23 12:52:28.174550 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 17/35] 10.222.0.30 () {64 vars in 1332 bytes} [Mon May 23 12:52:28 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:52:52.407869 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-307353d6-a0f6-426e-88d9-951b6a811032 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:52.409485 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-307353d6-a0f6-426e-88d9-951b6a811032 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001998 May 23 12:52:52.409934 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 19/36] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:52:52 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:52.420473 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:52.468280 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-f9b0a1bb-a269-4247-932d-4a66c876f3b0 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:52.485726 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-f9b0a1bb-a269-4247-932d-4a66c876f3b0 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.069738 May 23 12:52:52.486287 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 18/37] 10.222.0.30 () {60 vars in 1291 bytes} [Mon May 23 12:52:52 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:53.476361 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-307353d6-a0f6-426e-88d9-951b6a811032 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:53.477348 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-307353d6-a0f6-426e-88d9-951b6a811032 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001395 May 23 12:52:53.477746 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 20/38] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:52:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:53.695610 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-6e956c8e-cc87-4a9e-9e65-52bb9826a1e7 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:53.700965 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.wsgi [None req-6e956c8e-cc87-4a9e-9e65-52bb9826a1e7 admin admin] HTTP exception thrown: Flavor 42 could not be found. May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-6e956c8e-cc87-4a9e-9e65-52bb9826a1e7 admin admin] Request method failure captured: May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Accept: application/json May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Accept-Encoding: gzip, deflate May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Connection: keep-alive May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Content-Length: 0 May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Host: 10.222.0.30 May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Openstack-Api-Version: compute 2.90 May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Openstack-System-Scope: None May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: User-Agent: python-novaclient May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Auth-Token: gAAAAABii4OlrNGhicxUrZsYRDObBgZ5QZf44wFzZJEaieOLIQF23I_u7EHXp7idSfiE8Vcukrqf-yhPixiiyaN6KeahesDKso5pGAlM9VXP7_-gD3Je4TSnrNufd8Iky0KDWqBk3NcvCVF5tkyoEn6SeE0swWpeySld2sEddRQMX5sAS0W15DM May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Domain-Id: None May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Domain-Name: None May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Identity-Status: Confirmed May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Is-Admin-Project: True May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Openstack-Nova-Api-Version: 2.90 May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Domain-Id: default May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Domain-Name: Default May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Name: admin May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Role: member,reader,admin May 23 12:52:53.704905 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Roles: member,reader,admin May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant: admin May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant-Name: admin May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User: admin May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Domain-Id: default May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Domain-Name: Default May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Id: 0f7119f8fa5c4445977671140ded91a4 May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Name: admin May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: method: > May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: exception: Flavor 42 could not be found. May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: action_args: {'id': '42'} May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 12:52:53.706647 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:53.707609 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:53.708759 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-6e956c8e-cc87-4a9e-9e65-52bb9826a1e7 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=115197) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 12:52:53.709790 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-6e956c8e-cc87-4a9e-9e65-52bb9826a1e7 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.063853 May 23 12:52:53.709790 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 19/39] 10.222.0.30 () {60 vars in 1190 bytes} [Mon May 23 12:52:53 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 12:52:53.718278 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-68304027-7072-49f4-9682-4820491093d1 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:53.721217 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:53.739589 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-68304027-7072-49f4-9682-4820491093d1 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024941 May 23 12:52:53.740152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 21/40] 10.222.0.30 () {60 vars in 1210 bytes} [Mon May 23 12:52:53 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:53.750377 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-982a3570-425b-44a9-bc60-0d0e7459162d admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:53.772131 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-982a3570-425b-44a9-bc60-0d0e7459162d admin admin] 10.222.0.30 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025990 May 23 12:52:53.772656 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 20/41] 10.222.0.30 () {60 vars in 1210 bytes} [Mon May 23 12:52:53 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:53.782566 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-1e7bf70e-4762-4c85-8564-13e05ccef66b admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:53.788558 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.wsgi [None req-1e7bf70e-4762-4c85-8564-13e05ccef66b admin admin] HTTP exception thrown: Flavor 42 could not be found. May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-1e7bf70e-4762-4c85-8564-13e05ccef66b admin admin] Request method failure captured: May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Accept: application/json May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Accept-Encoding: gzip, deflate May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Connection: keep-alive May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Content-Length: 0 May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Host: 10.222.0.30 May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Openstack-Api-Version: compute 2.90 May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Openstack-System-Scope: None May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: User-Agent: python-novaclient May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Auth-Token: gAAAAABii4OlrNGhicxUrZsYRDObBgZ5QZf44wFzZJEaieOLIQF23I_u7EHXp7idSfiE8Vcukrqf-yhPixiiyaN6KeahesDKso5pGAlM9VXP7_-gD3Je4TSnrNufd8Iky0KDWqBk3NcvCVF5tkyoEn6SeE0swWpeySld2sEddRQMX5sAS0W15DM May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Domain-Id: None May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Domain-Name: None May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Identity-Status: Confirmed May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Is-Admin-Project: True May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Openstack-Nova-Api-Version: 2.90 May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Domain-Id: default May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Domain-Name: Default May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Name: admin May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Role: member,reader,admin May 23 12:52:53.791791 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Roles: member,reader,admin May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant: admin May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant-Name: admin May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User: admin May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Domain-Id: default May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Domain-Name: Default May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Id: 0f7119f8fa5c4445977671140ded91a4 May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Name: admin May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: method: > May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: exception: Flavor 42 could not be found. May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: action_args: {'id': '42'} May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 12:52:53.797983 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:53.798896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:53.799987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-1e7bf70e-4762-4c85-8564-13e05ccef66b admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=115196) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 12:52:53.801012 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-1e7bf70e-4762-4c85-8564-13e05ccef66b admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014697 May 23 12:52:53.801012 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 22/42] 10.222.0.30 () {60 vars in 1190 bytes} [Mon May 23 12:52:53 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) May 23 12:52:54.651496 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-982a3570-425b-44a9-bc60-0d0e7459162d admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:54.652502 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-982a3570-425b-44a9-bc60-0d0e7459162d admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001685 May 23 12:52:54.652899 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 21/43] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:52:54 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:54.846257 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-0b41fa32-878e-416f-a354-e832889221d9 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:54.851664 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.wsgi [None req-0b41fa32-878e-416f-a354-e832889221d9 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-0b41fa32-878e-416f-a354-e832889221d9 admin admin] Request method failure captured: May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Accept: application/json May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Accept-Encoding: gzip, deflate May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Connection: keep-alive May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Content-Length: 0 May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Host: 10.222.0.30 May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Openstack-Api-Version: compute 2.90 May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Openstack-System-Scope: None May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: User-Agent: python-novaclient May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Auth-Token: gAAAAABii4Omz64QUm4jps-ueop0KOaEZjS6ZNbvRDb13RSq7PPdLPUXnrMBEGvPj1Dn1YlOh5g5H8DCNKqkiQN-e1Eq-XdQ5ivpIHV7yNxyZMuYz30rcmm3espjNe4uHT6q2NLAx3Mepa6PeFuuze3_559vNbmj9hqgZCwdAP5Kpft9c2rDBgo May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Domain-Id: None May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Domain-Name: None May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Identity-Status: Confirmed May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Is-Admin-Project: True May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Openstack-Nova-Api-Version: 2.90 May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Domain-Id: default May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Domain-Name: Default May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Name: admin May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Role: member,reader,admin May 23 12:52:54.853152 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Roles: member,reader,admin May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant: admin May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant-Name: admin May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User: admin May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Domain-Id: default May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Domain-Name: Default May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Id: 0f7119f8fa5c4445977671140ded91a4 May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Name: admin May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: method: > May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: exception: Flavor 84 could not be found. May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: action_args: {'id': '84'} May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 12:52:54.854875 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:54.855887 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:54.857316 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-0b41fa32-878e-416f-a354-e832889221d9 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=115196) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 12:52:54.860616 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-0b41fa32-878e-416f-a354-e832889221d9 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.062420 May 23 12:52:54.860616 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 23/44] 10.222.0.30 () {60 vars in 1190 bytes} [Mon May 23 12:52:54 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 12:52:54.865578 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-3de3e3a4-7585-47f9-80f1-cd59f5bdb6eb admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:54.880737 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-3de3e3a4-7585-47f9-80f1-cd59f5bdb6eb admin admin] 10.222.0.30 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018799 May 23 12:52:54.881298 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 22/45] 10.222.0.30 () {60 vars in 1210 bytes} [Mon May 23 12:52:54 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:54.891296 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-d57dc2da-4dc1-4d16-a315-81282d463314 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:54.905987 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-d57dc2da-4dc1-4d16-a315-81282d463314 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018178 May 23 12:52:54.906506 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 24/46] 10.222.0.30 () {60 vars in 1210 bytes} [Mon May 23 12:52:54 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:54.915569 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-21365150-271d-4f4c-aef4-e11b6b208189 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:54.921409 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.wsgi [None req-21365150-271d-4f4c-aef4-e11b6b208189 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-21365150-271d-4f4c-aef4-e11b6b208189 admin admin] Request method failure captured: May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Accept: application/json May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Accept-Encoding: gzip, deflate May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Connection: keep-alive May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Content-Length: 0 May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Host: 10.222.0.30 May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Openstack-Api-Version: compute 2.90 May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Openstack-System-Scope: None May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: User-Agent: python-novaclient May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Auth-Token: gAAAAABii4Omz64QUm4jps-ueop0KOaEZjS6ZNbvRDb13RSq7PPdLPUXnrMBEGvPj1Dn1YlOh5g5H8DCNKqkiQN-e1Eq-XdQ5ivpIHV7yNxyZMuYz30rcmm3espjNe4uHT6q2NLAx3Mepa6PeFuuze3_559vNbmj9hqgZCwdAP5Kpft9c2rDBgo May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Domain-Id: None May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Domain-Name: None May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Identity-Status: Confirmed May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Is-Admin-Project: True May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Openstack-Nova-Api-Version: 2.90 May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Domain-Id: default May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Domain-Name: Default May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Name: admin May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Role: member,reader,admin May 23 12:52:54.922933 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Roles: member,reader,admin May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant: admin May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant-Name: admin May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User: admin May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Domain-Id: default May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Domain-Name: Default May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Id: 0f7119f8fa5c4445977671140ded91a4 May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Name: admin May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: method: > May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: exception: Flavor 84 could not be found. May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: action_args: {'id': '84'} May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 12:52:54.924494 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:54.925350 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:54.926487 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-21365150-271d-4f4c-aef4-e11b6b208189 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=115197) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 12:52:54.927545 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-21365150-271d-4f4c-aef4-e11b6b208189 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012179 May 23 12:52:54.927545 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 23/47] 10.222.0.30 () {60 vars in 1190 bytes} [Mon May 23 12:52:54 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 12:52:55.815373 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-d57dc2da-4dc1-4d16-a315-81282d463314 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:55.816368 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-d57dc2da-4dc1-4d16-a315-81282d463314 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001409 May 23 12:52:55.816761 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 25/48] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:52:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 12:52:56.007608 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b67b2d0d-7978-4b04-965f-3b3519a58bf7 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:56.012916 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.wsgi [None req-b67b2d0d-7978-4b04-965f-3b3519a58bf7 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b67b2d0d-7978-4b04-965f-3b3519a58bf7 admin admin] Request method failure captured: May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Accept: application/json May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Accept-Encoding: gzip, deflate May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Connection: keep-alive May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Content-Length: 0 May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Host: 10.222.0.30 May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Openstack-Api-Version: compute 2.90 May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Openstack-System-Scope: None May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: User-Agent: python-novaclient May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Auth-Token: gAAAAABii4OnegPSmitcw_og1I7zZThjkEyP9J8jI6b_R4ViAk-KZaDiebAHpsJd7oSExq9cLQtA5wgdm9-tDN6L1MAMRfVt0iIPevy51mxHlPQejaSels_FjgI0CFo9zS1qvZdqguoW1lkHKLJDwbo9zQKmrUXpQLPNxKvd96TbqdK_10rmxWA May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Domain-Id: None May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Domain-Name: None May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Identity-Status: Confirmed May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Is-Admin-Project: True May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Openstack-Nova-Api-Version: 2.90 May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Domain-Id: default May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Domain-Name: Default May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Project-Name: admin May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Role: member,reader,admin May 23 12:52:56.014781 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Roles: member,reader,admin May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant: admin May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-Tenant-Name: admin May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User: admin May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Domain-Id: default May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Domain-Name: Default May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Id: 0f7119f8fa5c4445977671140ded91a4 May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: X-User-Name: admin May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: method: > May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: exception: Flavor 451 could not be found. May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: action_args: {'id': '451'} May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 12:52:56.016341 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:56.017512 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: ERROR nova.api.openstack.wsgi May 23 12:52:56.018929 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-b67b2d0d-7978-4b04-965f-3b3519a58bf7 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=115197) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 12:52:56.020195 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-b67b2d0d-7978-4b04-965f-3b3519a58bf7 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.059565 May 23 12:52:56.020195 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 24/49] 10.222.0.30 () {60 vars in 1193 bytes} [Mon May 23 12:52:55 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 12:52:56.026548 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-dac82f59-c5dc-4320-81c5-109a9a817859 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:56.042376 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-dac82f59-c5dc-4320-81c5-109a9a817859 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019610 May 23 12:52:56.042900 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 26/50] 10.222.0.30 () {60 vars in 1210 bytes} [Mon May 23 12:52:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:56.053293 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:56.066431 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017123 May 23 12:52:56.066949 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 25/51] 10.222.0.30 () {60 vars in 1210 bytes} [Mon May 23 12:52:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:52:56.075896 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-a2226963-7776-43cb-b978-fa61a9334d1c admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:56.081556 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.wsgi [None req-a2226963-7776-43cb-b978-fa61a9334d1c admin admin] HTTP exception thrown: Flavor 451 could not be found. May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-a2226963-7776-43cb-b978-fa61a9334d1c admin admin] Request method failure captured: May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Accept: application/json May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Accept-Encoding: gzip, deflate May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Connection: keep-alive May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Content-Length: 0 May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Host: 10.222.0.30 May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Openstack-Api-Version: compute 2.90 May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Openstack-System-Scope: None May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: User-Agent: python-novaclient May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Auth-Token: gAAAAABii4OnegPSmitcw_og1I7zZThjkEyP9J8jI6b_R4ViAk-KZaDiebAHpsJd7oSExq9cLQtA5wgdm9-tDN6L1MAMRfVt0iIPevy51mxHlPQejaSels_FjgI0CFo9zS1qvZdqguoW1lkHKLJDwbo9zQKmrUXpQLPNxKvd96TbqdK_10rmxWA May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Domain-Id: None May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Domain-Name: None May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Identity-Status: Confirmed May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Is-Admin-Project: True May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Openstack-Nova-Api-Version: 2.90 May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Domain-Id: default May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Domain-Name: Default May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Project-Name: admin May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Role: member,reader,admin May 23 12:52:56.083053 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Roles: member,reader,admin May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/af8b84d1e3e44efba851eab6bed63282"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant: admin May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant-Id: af8b84d1e3e44efba851eab6bed63282 May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-Tenant-Name: admin May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User: admin May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Domain-Id: default May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Domain-Name: Default May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Id: 0f7119f8fa5c4445977671140ded91a4 May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: X-User-Name: admin May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: method: > May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: exception: Flavor 451 could not be found. May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: action_args: {'id': '451'} May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 12:52:56.084706 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:56.085653 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: ERROR nova.api.openstack.wsgi May 23 12:52:56.086816 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-a2226963-7776-43cb-b978-fa61a9334d1c admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=115196) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 12:52:56.087868 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-a2226963-7776-43cb-b978-fa61a9334d1c admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012044 May 23 12:52:56.087868 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 27/52] 10.222.0.30 () {60 vars in 1193 bytes} [Mon May 23 12:52:56 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 12:52:56.949131 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:56.950105 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001422 May 23 12:52:56.950509 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 26/53] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:52:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:57.145264 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-906714c7-5f2c-455d-9914-1323dff6a0c4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:57.242524 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-906714c7-5f2c-455d-9914-1323dff6a0c4 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.149132 May 23 12:52:57.242966 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 28/54] 10.222.0.30 () {64 vars in 1235 bytes} [Mon May 23 12:52:57 2022] POST /compute/v2.1/flavors => generated 404 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:58.155249 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:58.156277 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001441 May 23 12:52:58.156668 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 27/55] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:52:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:58.346271 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-3fe7058d-e20a-4488-82ad-834f5f225903 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:58.443113 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-3fe7058d-e20a-4488-82ad-834f5f225903 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.148812 May 23 12:52:58.443672 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 29/56] 10.222.0.30 () {64 vars in 1235 bytes} [Mon May 23 12:52:58 2022] POST /compute/v2.1/flavors => generated 405 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:52:59.301038 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:52:59.302095 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 May 23 12:52:59.302495 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 28/57] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:52:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:52:59.495018 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-0c6ded3b-2952-4e40-a06b-9f70ae864ee6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:52:59.568946 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-0c6ded3b-2952-4e40-a06b-9f70ae864ee6 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.125189 May 23 12:52:59.569412 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 30/58] 10.222.0.30 () {64 vars in 1235 bytes} [Mon May 23 12:52:59 2022] POST /compute/v2.1/flavors => generated 407 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:53:00.444339 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:53:00.444822 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001708 May 23 12:53:00.445219 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 29/59] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:53:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:53:00.636002 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-4dd9861a-7693-4b6a-a0d3-9793ebaa3ac2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:53:00.711532 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-4dd9861a-7693-4b6a-a0d3-9793ebaa3ac2 admin admin] 10.222.0.30 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.127398 May 23 12:53:00.712147 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 31/60] 10.222.0.30 () {64 vars in 1235 bytes} [Mon May 23 12:53:00 2022] POST /compute/v2.1/flavors => generated 409 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:53:01.580317 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:53:01.581320 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001418 May 23 12:53:01.581747 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 30/61] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:53:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:53:01.777239 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-d47dcad9-bcec-42f3-b4d9-21acc5d6402e admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:53:01.802708 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-d47dcad9-bcec-42f3-b4d9-21acc5d6402e admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.074943 May 23 12:53:01.802708 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 32/62] 10.222.0.30 () {60 vars in 1202 bytes} [Mon May 23 12:53:01 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:54:03.738351 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:54:03.739340 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-496fad77-9f1f-4fbe-b183-0a7f2c7eb38f admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 May 23 12:54:03.739733 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 31/63] 10.222.0.30 () {54 vars in 930 bytes} [Mon May 23 12:54:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:54:03.747464 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:54:03.802211 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-1354e6ab-0fa7-48af-80f7-db04f93a00d5 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:54:03.822536 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-1354e6ab-0fa7-48af-80f7-db04f93a00d5 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.076491 May 23 12:54:03.823093 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 33/64] 10.222.0.30 () {60 vars in 1291 bytes} [Mon May 23 12:54:03 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:54:03.843884 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG dbcounter [-] [115197] Writing DB stats nova_api:SELECT=14 {{(pid=115197) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:54:03.847572 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-e03fbf06-d59a-4833-a340-4d8e2aa43ff8 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:54:03.855557 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-e03fbf06-d59a-4833-a340-4d8e2aa43ff8 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013081 May 23 12:54:03.856072 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 32/65] 10.222.0.30 () {60 vars in 1282 bytes} [Mon May 23 12:54:03 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:54:03.867365 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-6f9c1946-d0ce-410b-acda-92138f3299d9 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:54:03.876012 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-6f9c1946-d0ce-410b-acda-92138f3299d9 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012285 May 23 12:54:03.881752 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 34/66] 10.222.0.30 () {60 vars in 1285 bytes} [Mon May 23 12:54:03 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:54:03.887919 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:54:03.895592 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011487 May 23 12:54:03.896102 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 33/67] 10.222.0.30 () {60 vars in 1285 bytes} [Mon May 23 12:54:03 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:54:03.908780 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-e535cf60-0359-4386-b76e-f1dc954483e6 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:54:03.916872 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-e535cf60-0359-4386-b76e-f1dc954483e6 admin admin] 10.222.0.30 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011029 May 23 12:54:03.917943 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 35/68] 10.222.0.30 () {60 vars in 1282 bytes} [Mon May 23 12:54:03 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 12:57:48.687120 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:57:48.688190 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001571 May 23 12:57:48.688674 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 34/69] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:57:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 12:57:48.843357 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_api:SELECT=6 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:57:48.956496 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-b77a1d41-305b-4c9d-8146-78e9a04d73e0 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:57:48.957651 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:57:48.957651 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: warnings.warn(msg) May 23 12:57:48.958902 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-b77a1d41-305b-4c9d-8146-78e9a04d73e0 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:57:48.959336 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-b77a1d41-305b-4c9d-8146-78e9a04d73e0 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:57:48.960111 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-b77a1d41-305b-4c9d-8146-78e9a04d73e0 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:57:48.960529 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-b77a1d41-305b-4c9d-8146-78e9a04d73e0 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:57:48.993980 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-b77a1d41-305b-4c9d-8146-78e9a04d73e0 admin admin] 10.222.0.30 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.152612 May 23 12:57:48.994536 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 36/70] 10.222.0.30 () {60 vars in 1193 bytes} [Mon May 23 12:57:48 2022] GET /compute/v2.1/os-services => generated 791 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:57:49.866547 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:57:49.867586 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001562 May 23 12:57:49.867986 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 35/71] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:57:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:57:50.063114 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-207b938a-3b08-4921-932f-6783267c7c79 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:57:50.064300 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:57:50.064300 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: warnings.warn(msg) May 23 12:57:50.065515 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-207b938a-3b08-4921-932f-6783267c7c79 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:57:50.065996 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-207b938a-3b08-4921-932f-6783267c7c79 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:57:50.066805 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-207b938a-3b08-4921-932f-6783267c7c79 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:57:50.067256 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-207b938a-3b08-4921-932f-6783267c7c79 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:57:50.090812 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-207b938a-3b08-4921-932f-6783267c7c79 admin admin] 10.222.0.30 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.077325 May 23 12:57:50.090812 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 37/72] 10.222.0.30 () {60 vars in 1193 bytes} [Mon May 23 12:57:50 2022] GET /compute/v2.1/os-services => generated 791 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 12:58:01.406002 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:58:01.407013 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002453 May 23 12:58:01.407408 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 36/73] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:58:01 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:58:01.558156 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_cell0:SELECT=4 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:58:01.558871 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_cell1:SELECT=4 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:58:01.559652 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG dbcounter [-] [115196] Writing DB stats nova_api:SELECT=8 {{(pid=115196) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:58:01.608937 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-d00f9d7e-3f36-4f49-aa1c-b07775702936 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:58:01.610119 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:58:01.610119 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: warnings.warn(msg) May 23 12:58:01.611293 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-d00f9d7e-3f36-4f49-aa1c-b07775702936 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:58:01.611720 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-d00f9d7e-3f36-4f49-aa1c-b07775702936 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:58:01.612494 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-d00f9d7e-3f36-4f49-aa1c-b07775702936 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:58:01.612910 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-d00f9d7e-3f36-4f49-aa1c-b07775702936 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:58:01.638803 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-d00f9d7e-3f36-4f49-aa1c-b07775702936 admin admin] 10.222.0.30 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.085025 May 23 12:58:01.639880 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 38/74] 10.222.0.30 () {60 vars in 1193 bytes} [Mon May 23 12:58:01 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:58:02.516674 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: DEBUG nova.api.openstack.wsgi [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] Calling method '>' {{(pid=115197) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:58:02.518464 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: INFO nova.api.openstack.requestlog [None req-617bbf90-afa9-4c83-ba80-631c647dc5e3 admin admin] 10.222.0.30 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002246 May 23 12:58:02.518856 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: [pid: 115197|app: 0|req: 37/75] 10.222.0.30 () {56 vars in 1074 bytes} [Mon May 23 12:58:02 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:58:02.708449 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG nova.api.openstack.wsgi [None req-1dfb1999-265a-4e2a-bd7a-ee8b8b03b3f2 admin admin] Calling method '>' {{(pid=115196) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:58:02.709623 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:58:02.709623 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: warnings.warn(msg) May 23 12:58:02.710846 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-1dfb1999-265a-4e2a-bd7a-ee8b8b03b3f2 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:58:02.711280 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-1dfb1999-265a-4e2a-bd7a-ee8b8b03b3f2 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:58:02.712061 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-1dfb1999-265a-4e2a-bd7a-ee8b8b03b3f2 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:58:02.712494 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: DEBUG oslo_concurrency.lockutils [None req-1dfb1999-265a-4e2a-bd7a-ee8b8b03b3f2 admin admin] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115196) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:58:02.734767 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: INFO nova.api.openstack.requestlog [None req-1dfb1999-265a-4e2a-bd7a-ee8b8b03b3f2 admin admin] 10.222.0.30 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.078423 May 23 12:58:02.735243 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: [pid: 115196|app: 0|req: 39/76] 10.222.0.30 () {60 vars in 1193 bytes} [Mon May 23 12:58:02 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 12:59:04.049157 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... May 23 12:59:04.050025 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: Mon May 23 12:59:04 2022 - graceful shutdown triggered... May 23 12:59:04.050544 dv-cinder-master-3-iscsi devstack@n-api.service[115196]: Gracefully killing worker 1 (pid: 115196)... May 23 12:59:04.050913 dv-cinder-master-3-iscsi devstack@n-api.service[115197]: Gracefully killing worker 2 (pid: 115197)... May 23 12:59:05.051247 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: worker 1 buried after 1 seconds May 23 12:59:05.051247 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: worker 2 buried after 1 seconds May 23 12:59:05.051247 dv-cinder-master-3-iscsi devstack@n-api.service[115195]: goodbye to uWSGI. May 23 12:59:05.052798 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. May 23 12:59:05.053360 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. May 23 12:59:05.056202 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 23 12:59:05.074965 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 12:59:05 2022] *** May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: nodename: dv-cinder-master-3-iscsi May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: machine: x86_64 May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: clock source: unix May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: pcre jit disabled May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: detected number of CPU cores: 4 May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: current working directory: / May 23 12:59:05.094413 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: detected binary path: /usr/bin/uwsgi-core May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: your processes number limit is 47680 May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: your memory page size is 4096 bytes May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: detected max file descriptor number: 2048 May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: lock engine: pthread robust mutexes May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: thunder lock: enabled May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 23 12:59:05.095209 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 12:59:05.142804 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: Python main interpreter initialized at 0x55e67f4e29c0 May 23 12:59:05.142804 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: python threads support enabled May 23 12:59:05.142804 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: your server socket listen backlog is limited to 100 connections May 23 12:59:05.142804 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: your mercy for graceful operations on workers is 90 seconds May 23 12:59:05.143239 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: mapped 403077 bytes (393 KB) for 2 cores May 23 12:59:05.143239 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: *** Operational MODE: preforking *** May 23 12:59:05.143489 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: *** uWSGI is running in multiple interpreter mode *** May 23 12:59:05.143489 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: spawned uWSGI master process (pid: 130032) May 23 12:59:05.143699 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 23 12:59:05.144273 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: spawned uWSGI worker 1 (pid: 130045, cores: 1) May 23 12:59:05.145071 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: spawned uWSGI worker 2 (pid: 130046, cores: 1) May 23 12:59:05.145723 dv-cinder-master-3-iscsi devstack@n-api.service[130032]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 12:59:06.327616 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 12:59:06.328195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 12:59:07.110700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:59:07.110700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: from cryptography.utils import int_from_bytes May 23 12:59:07.111171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:59:07.111171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: from cryptography.utils import int_from_bytes May 23 12:59:07.131221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:59:07.131221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: from cryptography.utils import int_from_bytes May 23 12:59:07.131561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:59:07.131561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: from cryptography.utils import int_from_bytes May 23 12:59:09.534803 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 12:59:09.535431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 12:59:09.535741 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 12:59:09.535949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 12:59:09.536157 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 12:59:09.536540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.536865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.537183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.537501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.537828 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.538162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.538480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.538795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.539119 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.539382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.539805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.540132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.540453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.540761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.541068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.541382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.541682 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.542033 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.542338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.542839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.544323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.544323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.544323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.544323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.544670 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.544912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.545149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.545457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.545772 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.546096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.546489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.546818 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.547135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.548527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.548527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.548527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.548527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.549418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.549418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.549418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.550068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.550068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.550523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.550745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.550929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.551182 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.551425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.551744 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.552052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.552353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.552600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.552906 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.553226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.553544 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.553859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.554136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.554387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.554611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 12:59:09.555099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.555258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 12:59:09.555410 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.555410 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.555706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 12:59:09.555886 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.556044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 12:59:09.556199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.556344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 12:59:09.556580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.556739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.556885 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.557043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.557242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.557391 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.557546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.557782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.557978 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.558147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.558305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.558457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.558614 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.558761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.560616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.560616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.561104 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.561277 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.561436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.561583 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.30 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.561751 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.561920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.30 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562139 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562400 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.562972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563883 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.563986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564075 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564259 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564661 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564661 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.564913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565686 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565783 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.565870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.566851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567148 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567345 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567643 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.567945 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.568073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.568245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.568387 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.568546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.568696 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.568859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.569019 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.569161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.569342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.569492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.569654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.569749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.570951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.571084 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.571250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.571401 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.571559 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.571866 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572465 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.572913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573575 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.573929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574055 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574365 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.574864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.575080 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.575189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.575403 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.575512 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.576376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.576498 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.576608 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.576702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.576919 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.577084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.577227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.577390 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.577536 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.577693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.577845 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578016 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578655 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578807 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.578967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.579129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.579292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.579443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.579584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.579745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.579852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.580066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.580173 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.580462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.30 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.580564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.580713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.30 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.580817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.581021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.581117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.581360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.581456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.581679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.581782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.582010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.582126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.582329 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.582425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.582647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.582745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.582955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.583103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.583264 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.583368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.583574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.583720 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.583879 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.584043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.584249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.584386 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.584486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.584575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.584740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.584843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.585040 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.585141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.585344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.585448 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.585666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.585782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.585989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.586099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.586296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.586436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.586606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.586731 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.586904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.587035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.587213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.587342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.587514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.587637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.587819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.587947 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.588125 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.588257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.588428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.588557 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.588730 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.588862 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.589035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.589171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.589337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.589459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.589640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.589785 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.589981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.590128 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.590303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.590438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.590610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.590738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.590912 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.591050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.591216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.591361 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.591566 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.591689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.591887 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.592018 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.592193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.592327 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.592497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.592620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.592816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.592956 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.593122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.593245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.593426 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.593548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.593759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.594133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.594294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.594513 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.594707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.594808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.594902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.595006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.595108 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.595203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.595297 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.595394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.595545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.595650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.595848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.595954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.596151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.596261 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.596461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.596605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.596746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.596884 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.597032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.597175 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.597328 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.597485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.597628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.597760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.597926 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.598058 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.598215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.598312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.598531 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.598636 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.598853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.598950 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.599162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.599277 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.599464 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.599577 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.599767 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.599876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.600067 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.600170 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:59:09.600389 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.600493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.600697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.600792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.600998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.601102 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.601316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.601424 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.601618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.601721 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.601942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.602964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.603175 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.603271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.603481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.603576 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.603814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.603968 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.604118 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.604255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.604424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.604642 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.604749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.604844 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.605056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.605161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.605357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.605459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.605676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.605782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.606014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.606112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.606333 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.606429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.606652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.606767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.607896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608000 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.608908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.609105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.609212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.609403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.609506 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.609701 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.609804 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610362 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.610965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.611097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.611276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.611418 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.611593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.611732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.611895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612047 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612878 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.612984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.613199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.613307 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.613507 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.613610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.613818 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.613951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.614159 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.614260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.614471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.614633 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.614835 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.614948 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.615073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.615169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.615388 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.615483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.615692 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.615797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.616009 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.616110 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.616314 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.616409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.616621 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.616929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617061 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617566 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617686 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.617991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.618221 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.618316 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.618528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.618632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.618841 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.618945 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.619153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.619249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.619466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.619574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.619772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.619876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620081 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620198 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620651 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.620998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.621132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.621304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.621402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.622052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.622231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.622231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.622231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.625053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.625251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.625411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.625564 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.625727 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.625951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.626123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.626271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.626425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.626571 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.626715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.626861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627015 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627126 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627399 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.627950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628148 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628336 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628525 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628627 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628903 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.628997 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629287 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629565 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629651 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.629962 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.630970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.631151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.631244 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.631460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.631574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.631766 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.631932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632067 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632170 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.632990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.633091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.633291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.633386 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.633594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.633697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.633940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.634043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.634262 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.634407 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.634570 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.634717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.634871 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635284 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.635951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.636124 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.636231 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.636431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.636540 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.636736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.636832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637049 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637143 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637358 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637664 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.637994 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.638093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.638379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.638481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.638691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.638917 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.639002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.639304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.639408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.639609 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.639702 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.639915 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.640017 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.640477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.640588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.640832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.640933 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.641166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.641258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.641483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.641581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.641807 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.642088 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.642186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.642419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.642527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.642731 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.642877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643140 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643763 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.643987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.644089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.644299 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.644408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.644619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.644757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.644927 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.645959 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.646164 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.646270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.646485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.646624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.646791 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.646919 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.647981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.648994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.649190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.649294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.649496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.649593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.649795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.649935 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.650133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.650237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.650441 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.650536 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.650745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.650839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.651969 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.652095 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.652268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.652374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.652581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.652685 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.652904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.653011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.653217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.653320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.653517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.653612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.654381 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.654488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.654706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.654890 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.655014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.655423 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.655541 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.655640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.655727 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.655935 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656242 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656339 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.656952 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.657164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.657268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.657473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.657580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.657781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.657898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.658109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.658205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.658419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.658538 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.658717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.658835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.659935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660244 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660849 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.660980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.661155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.661284 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.661453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.661590 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.661765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.661929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662094 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.662999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.663127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.663303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.663424 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.663601 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.663733 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.663922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.664182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.664318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.664518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.664648 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.664816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.664937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.665121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.665294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.665411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.665541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.665761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.665914 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666351 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666455 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666668 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.666967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.667073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.667276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.667405 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.667577 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.667681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.667880 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668084 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668291 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668475 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.668874 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.669076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.669182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.669377 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.669530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.669775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.669982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.670106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.670203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.670413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.670503 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.670723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.670817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671434 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.671933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672274 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.672955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.673155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.673259 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.673459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.673554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.673797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.673955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.674127 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.674233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.674452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.674558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.674756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.674850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.675068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.675165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.675375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.675527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.675686 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.675932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676055 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676170 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.676890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.677020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.677199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.677337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.677509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.677645 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.677809 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678148 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678495 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678616 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.678853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.679941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680248 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.680991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.681159 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.681290 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.681464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.681593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.681769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.681924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.682103 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.682243 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.682406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.682561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.682707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.682828 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683022 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683145 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.683925 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684359 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.684954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.685781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.685937 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.686153 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.686393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.686524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.686662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.686788 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.686902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.687968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.688090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.688270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.688402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.688594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.688717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.688900 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.689991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.690160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.690307 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.690466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.690588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.690769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.690892 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691207 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.691970 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.692100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.692277 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.692407 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.692589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.692719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.692893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.693048 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.693182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.693321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.693488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.693693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.693838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.694152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.694295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.694462 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.694610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.694777 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.694923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.695097 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.695235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.695921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696804 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.696943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697087 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697770 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.697932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698067 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698188 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698790 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.698929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699556 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699673 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.699908 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.700229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.700432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.700605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.701369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.701574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.701843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.702132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.702282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.702443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.702591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.702735 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.702880 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.703920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.704085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.704233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.704377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.704531 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.704687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.704844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.705155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.705471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.705624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.705770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.705986 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.706139 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.706310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.706470 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.706617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.706769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.706924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707379 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.707993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.708140 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.708305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.708461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.708623 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.708781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.708928 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.709073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.709219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.709363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.709523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.709672 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.709825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.710011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.710231 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.710554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.710996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.711159 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.711306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.711452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.711607 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.711764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.711921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.712068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.712222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.712396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.712506 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.712728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.712839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.713046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.713146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.713371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.713466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.713696 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.713793 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714333 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714430 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.714957 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715060 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715262 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.715974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.716185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.716483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.716600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.716741 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.716890 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717422 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717555 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717786 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.717964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.718079 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.718290 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.718431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.718616 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.718751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.718924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719214 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719316 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719616 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719821 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.719933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.720162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.720266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.720464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.720576 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.720779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.720884 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.721088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.721186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.721408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.721503 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.721732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.721837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.722074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.722180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.722396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.722490 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.722712 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.722824 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723124 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723655 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723751 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.723968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.724072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.724282 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.724387 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.724604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.724700 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.724913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725017 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725525 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725622 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.725966 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.726165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.726432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.726544 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.726765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.726862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.727079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.727182 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.727387 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.727526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.727691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.727827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728098 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728413 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.728931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.729035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.729242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.729374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.729552 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.729657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.729854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730021 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730208 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.730967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.731133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.731274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.731438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.731534 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.731761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.731865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.732615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.732747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.732942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.733234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.733391 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.733589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.733720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.734003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.734146 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.734269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.734395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.734643 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.734963 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.735278 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.735371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.735594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.735701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.735910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.735996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.736226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.736318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.736539 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.736639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.736850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.736936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.737166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.737481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.737808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.738126 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.738265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.738457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.738586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.738785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.738940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.739089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.739207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.739406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.739528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.739717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.739921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.740038 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.740242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.740373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.740577 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.740710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.740905 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.741028 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.741230 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.741352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.741546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.741684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.741862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.742041 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.742195 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.742317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.742504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.742640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.742854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743012 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743356 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743792 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.743921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.744112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.744241 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.744426 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.744549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.744729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.744868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.745056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.745185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.745364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.745486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.745683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.745998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.746158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.746305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.746469 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.746615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.746809 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.746961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.747114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.747261 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.747488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.747763 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.747921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.748074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.748220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.748378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.748533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.748688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.748843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.749085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.749245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.749402 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.749549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.749695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.749848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.750090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.750407 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.750724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.750932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.751085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.751239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.751396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.751549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.751706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.751853 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.752025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.752257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.752409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.752565 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.752713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.752858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753157 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753475 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753768 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.753971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.754138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.754286 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.754445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.754591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.754737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.754889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755351 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755804 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.755950 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.756109 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.756256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.756409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.756561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.756708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.756861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757184 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757484 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757636 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.757812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.758155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.758317 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.758470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.758470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.758769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.758923 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.759075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.759245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.759399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.759550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.759711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.759863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.760945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.761120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.761299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.761467 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.761737 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.761947 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.762110 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.762263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.762426 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.762594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.762754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.762908 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.763059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.763746 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.763978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.764186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.764356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.764510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.764662 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.764814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.764987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.765432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.765600 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.765755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.765932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.766090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.766266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.766428 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.766582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.766744 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.766898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767049 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.767995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.768162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.768332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.768487 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.768639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.768790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.768956 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.769118 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.769271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.769423 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.769576 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.769728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.769911 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.770071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.770233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.770385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.770597 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.770757 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.770921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.771074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.771225 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.771376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.771528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.771693 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.771845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.772942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.773094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.773839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.774052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.774220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.774382 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.774549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.774711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.774863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.775049 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.775211 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.775363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.775530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.775683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.775844 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776796 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.776973 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.777134 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.777287 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.777438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.777605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.777758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.777989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.778158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.778311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.778470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.778631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.778918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.779085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.779249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.779412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.779566 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.779726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.779893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780048 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780513 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780513 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.780982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.781141 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.781304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.781486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.781486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.781486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.783931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.784093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.784258 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.784412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.784596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.784596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.785356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.785533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.785710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.785865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786824 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.786984 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.787137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.787289 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.787440 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.787592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.787754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.787919 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.788090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.788252 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.788414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.788575 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.788736 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.788889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.789048 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.789201 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.789353 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.789512 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.789681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.789856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.790969 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.791121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.791272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.791437 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.791591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.791761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.791914 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.792066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.792225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.792392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.792546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.792698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.792857 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.793018 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.793178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.793543 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.793706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.793859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.794078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.794240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.794393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.794393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.795290 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.795451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.795617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.795848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.796024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.796178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.796330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.796482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.796705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.796864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797333 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797484 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797635 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797789 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.797975 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.798134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.798295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.798457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.798610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.798785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.798939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.799091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.799252 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.799252 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.799544 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.799710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.799865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.800025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.800200 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.800364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.800525 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.800697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.800851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801004 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801313 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.801991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.802150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.802312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.802475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.802640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.802792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.802944 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.803097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.803249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.803414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.803567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.803718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.803869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804505 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.804679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.805137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.805310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.805474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.805634 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.805787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.805787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130046) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.806132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.807303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.808462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.809722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.809722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.809722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.809722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.811027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.811208 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.811393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.811691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.812007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.812325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.812656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.812987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.813300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.813630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.813980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.814310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.814631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.814940 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.815273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.815590 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.815903 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.816222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.816536 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.816849 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.817167 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.817487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.817814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.818175 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.818491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.818819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.819148 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.819498 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.819817 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.820167 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.820502 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.820822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.821135 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.821440 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:59:09.821683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130045) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 12:59:09.863622 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO dbcounter [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Registered counter for database nova_api May 23 12:59:09.883025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO dbcounter [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Registered counter for database nova_api May 23 12:59:09.884703 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_db.sqlalchemy.engines [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130046) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:09.894098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writer thread running {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:09.903003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_db.sqlalchemy.engines [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130045) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:09.910531 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writer thread running {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:09.951560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.context [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3c6cf08e-95e3-44d9-8446-eef7c93b66c5(cell1) {{(pid=130046) load_cells /opt/stack/nova/nova/context.py:464}} May 23 12:59:09.955915 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:59:09.957530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:59:09.958370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:59:09.960399 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:59:09.962427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO dbcounter [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Registered counter for database nova_cell0 May 23 12:59:09.967419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO dbcounter [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Registered counter for database nova_cell1 May 23 12:59:09.969301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.context [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3c6cf08e-95e3-44d9-8446-eef7c93b66c5(cell1) {{(pid=130045) load_cells /opt/stack/nova/nova/context.py:464}} May 23 12:59:09.975499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:59:09.975675 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_db.sqlalchemy.engines [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130046) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:09.975777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:59:09.976213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:59:09.976579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_db.sqlalchemy.engines [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130046) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:09.978232 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:59:09.980331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO dbcounter [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Registered counter for database nova_cell0 May 23 12:59:09.981936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writer thread running {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:09.984347 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writer thread running {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:09.985105 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO dbcounter [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Registered counter for database nova_cell1 May 23 12:59:09.992456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_db.sqlalchemy.engines [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130045) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:09.995006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_db.sqlalchemy.engines [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130045) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:10.000266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writer thread running {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:10.003355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writer thread running {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:10.134121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO dbcounter [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Registered counter for database nova_cell0 May 23 12:59:10.141777 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_db.sqlalchemy.engines [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130046) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:10.146492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writer thread running {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:10.151129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO dbcounter [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Registered counter for database nova_cell0 May 23 12:59:10.158110 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_db.sqlalchemy.engines [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130045) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:59:10.162456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writer thread running {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:59:10.464300 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: WARNING keystonemiddleware.auth_token [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:59:10.472201 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: WARNING keystonemiddleware.auth_token [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:59:10.484138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: WARNING keystonemiddleware.auth_token [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:59:10.491895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: WARNING keystonemiddleware.auth_token [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:59:10.820017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: WARNING keystonemiddleware.auth_token [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:59:10.826565 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: WARNING keystonemiddleware.auth_token [None req-aee76fa3-8391-4728-ae22-4371cc5fdf8b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:59:10.832008 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e67f4e29c0 pid: 130046 (default app) May 23 12:59:10.839018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: WARNING keystonemiddleware.auth_token [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 12:59:10.845671 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: WARNING keystonemiddleware.auth_token [None req-9968295d-d54e-4bdf-babe-bfcd9f9c7314 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 12:59:10.850995 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e67f4e29c0 pid: 130045 (default app) May 23 12:59:54.037649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:54.037649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:54.042581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:54.044989 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:55.231737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dcab0989-0e44-4ab5-860e-24881dc7d540 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Action: 'create', calling method: .version_select at 0x7f9b18eccca0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1482492403"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:59:56.394304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:56.396676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:56.398464 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:56.399991 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:59:56.928380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-410662673", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "networks": [{"uuid": "50e5ce1d-72f1-4bb0-be55-af6d47fef78d"}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:59:57.476544 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] validate_networks() for [('50e5ce1d-72f1-4bb0-be55-af6d47fef78d', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 12:59:57.701516 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dcab0989-0e44-4ab5-860e-24881dc7d540 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 3.675414 May 23 12:59:57.702189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 1/1] 10.222.0.30 () {60 vars in 1154 bytes} [Mon May 23 12:59:54 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 3679 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 23 12:59:57.716743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b05c94cd-b422-4baf-9a63-6455f2937514 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18d1ec10>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:59:57.873562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b05c94cd-b422-4baf-9a63-6455f2937514 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.162637 May 23 12:59:57.874503 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 2/2] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 12:59:57 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:59:58.735302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-96db7f03-7185-4e31-ae03-2c7017df5950 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b18d1edc0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:59:58.823680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-96db7f03-7185-4e31-ae03-2c7017df5950 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/2222ef42-ac2c-4916-846f-0d37dde7d0c4" status: 200 len: 351 microversion: 2.1 time: 0.098211 May 23 12:59:58.826309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 3/3] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 12:59:58 2022] GET /compute/v2.1/os-volumes/2222ef42-ac2c-4916-846f-0d37dde7d0c4 => generated 351 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:59:58.890598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-63c388d1-6892-4f12-8847-6a003f4b635d tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18d33430>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:59:59.247786 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-63c388d1-6892-4f12-8847-6a003f4b635d tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 346 microversion: 2.1 time: 0.364877 May 23 12:59:59.248911 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 4/4] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 12:59:58 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 346 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 12:59:59.260031 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-782aa409-749d-4b7d-8fa2-51a854557a86 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Action: 'create', calling method: .version_select at 0x7f9b18d33a60>, body: {"snapshot": {"volume_id": "362e9bb0-0628-436a-b4cd-2711c6bf4cc4", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-879028879"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 12:59:59.318663 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 12:59:59.319241 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 12:59:59.319882 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 12:59:59.320471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 12:59:59.320904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 12:59:59.348784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:59:59.349377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:59:59.350271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 12:59:59.351292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 12:59:59.400256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Getting quotas for project 9e3c84deee9a435a9c1e72e8408c18b4. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 12:59:59.408979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Getting quotas for user a9b80726b3f14828b8815e30fc7e13c5 and project 9e3c84deee9a435a9c1e72e8408c18b4. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 12:59:59.426088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 12:59:59.429332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 12:59:59.687471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-643b9222-f2e5-457d-b92b-1c1820f3f1e1 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.306162 May 23 12:59:59.690440 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 1/5] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 12:59:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 3310 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 12:59:59.721715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b5ec2fe0-1f5c-48c8-a72f-ef03a4fe3ae7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 12:59:59.724216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b5ec2fe0-1f5c-48c8-a72f-ef03a4fe3ae7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 12:59:59.853181 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b5ec2fe0-1f5c-48c8-a72f-ef03a4fe3ae7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 12:59:59.883099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b5ec2fe0-1f5c-48c8-a72f-ef03a4fe3ae7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1214 microversion: 2.1 time: 0.179426 May 23 12:59:59.883526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 2/6] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 12:59:59 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1214 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:00.484343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-782aa409-749d-4b7d-8fa2-51a854557a86 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 1.227269 May 23 13:00:00.485872 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 5/7] 10.222.0.30 () {60 vars in 1161 bytes} [Mon May 23 12:59:59 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 1230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:00:00.497463 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f8aa0147-7063-460b-8a77-da5d01060187 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18ccaf70>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:00.823983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f8aa0147-7063-460b-8a77-da5d01060187 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.331558 May 23 13:00:00.825550 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 3/8] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:00 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:00.835494 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-da2f7177-c151-49c3-bf7f-f1a11fe7a091 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18d33dc0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:00.900431 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ab4d2142-f698-40fa-b662-818933a7a900 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:00.901153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ab4d2142-f698-40fa-b662-818933a7a900 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:00.934576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-da2f7177-c151-49c3-bf7f-f1a11fe7a091 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.100467 May 23 13:00:00.934576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 6/9] 10.222.0.30 () {58 vars in 1139 bytes} [Mon May 23 13:00:00 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:00.943630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b25a5e8c-b3b9-4afe-b382-98541ee97398 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18d1ea60>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:01.005930 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b25a5e8c-b3b9-4afe-b382-98541ee97398 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.066501 May 23 13:00:01.006790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 7/10] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:00 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:01.061465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ab4d2142-f698-40fa-b662-818933a7a900 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:01.069717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ab4d2142-f698-40fa-b662-818933a7a900 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1214 microversion: 2.1 time: 0.168738 May 23 13:00:01.070604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 4/11] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:00 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1214 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:01.650329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0b1540fe-b976-4cd4-9527-a4ed91aec04e tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b18d421f0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:01.721953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0b1540fe-b976-4cd4-9527-a4ed91aec04e tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/aac010dc-ef7a-4704-89aa-532c62417fca" status: 200 len: 351 microversion: 2.1 time: 0.077387 May 23 13:00:01.722604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 8/12] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:01 2022] GET /compute/v2.1/os-volumes/aac010dc-ef7a-4704-89aa-532c62417fca => generated 351 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:02.028729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1c59fe8c-1858-4149-8dae-cb73d3df16f5 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b183ea8b0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:02.090738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c11bca56-ae8c-43ab-b86c-73602f2e0ecc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:02.091573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c11bca56-ae8c-43ab-b86c-73602f2e0ecc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:02.107489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1c59fe8c-1858-4149-8dae-cb73d3df16f5 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.092251 May 23 13:00:02.108510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 5/13] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:02 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:02.244978 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c11bca56-ae8c-43ab-b86c-73602f2e0ecc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:02.280586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c11bca56-ae8c-43ab-b86c-73602f2e0ecc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1214 microversion: 2.1 time: 0.190651 May 23 13:00:02.280586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 9/14] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:02 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1214 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:03.127486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8b7e3c9d-0aec-45f3-a124-11b48752f206 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b183f9550>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:03.188077 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8b7e3c9d-0aec-45f3-a124-11b48752f206 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.072803 May 23 13:00:03.193052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 6/15] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:03 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:03.297528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:03.299235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:03.312535 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:03.317706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:03.317706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:03.317706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:03.990601 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:04.004156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7836445b-70a5-404f-b75d-764466222329 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1218 microversion: 2.1 time: 0.713325 May 23 13:00:04.005289 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 10/16] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:03 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1218 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:04.201824 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-30cf611d-6f54-4893-b47d-79e4639545de tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b183f9c10>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:04.249637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-30cf611d-6f54-4893-b47d-79e4639545de tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.053841 May 23 13:00:04.251894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 7/17] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:04 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:04.655433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2acec714-6762-4434-900c-2cea2493ac1f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: .version_select at 0x7f9b18c1e040>, body: {"keypair": {"name": "tempest-keypair-756628403", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLUvRwZiyxMkdmSr2NAIknmWW2yQAd9CHVu2UAVv0/pypl2qekXZV4WZ4wfHMIR7ga7GnfA0rqE/MUySVr3oKKDArcVrAVkvMIo6UkbDXuaqGQKSorQ984UvlhNarlIKYA=="}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:04.727284 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-2acec714-6762-4434-900c-2cea2493ac1f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:00:04.742050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-2acec714-6762-4434-900c-2cea2493ac1f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:00:04.881697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2acec714-6762-4434-900c-2cea2493ac1f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.294678 May 23 13:00:04.886238 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 11/18] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:00:04 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:05.023260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:05.023872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:05.036432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:05.036891 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:05.039879 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:05.040476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:05.274486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c2189815-dcec-4421-9aff-22696d43d50f tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18bbe4c0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:05.295191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:05.319724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c67dca1a-64b3-42e2-bee4-c3c5c2f362c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1218 microversion: 2.1 time: 0.296562 May 23 13:00:05.345520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 8/19] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:05 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1218 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:05.356660 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c2189815-dcec-4421-9aff-22696d43d50f tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.096025 May 23 13:00:05.357605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 12/20] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:05 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:05.851783 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-410256219", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:05.922764 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] validate_networks() for [('11cfb087-2166-41ce-9a36-a5b349287bd2', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:00:06.360472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:06.366373 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:06.395466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:06.396040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:06.399002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:06.400162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:06.435088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:00:06.435521 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:00:06.435964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:00:06.436371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:00:06.436780 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:00:06.458061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:06.458519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:06.459327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:06.459769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:06.507768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:00:06.508056 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:06.518469 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8ae64f79-652b-47bd-b2c6-d258ae685a4d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1218 microversion: 2.1 time: 0.172667 May 23 13:00:06.520690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 13/21] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:06 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1218 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:06.523097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for user b18e5f1b24634096995068e41d94cad6 and project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:00:06.528321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7872ad33-d3bf-4e12-96e6-03e22cce1573 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18bbedc0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:06.553838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:00:06.555989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:00:06.597456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7872ad33-d3bf-4e12-96e6-03e22cce1573 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.075200 May 23 13:00:06.600177 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 14/22] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:06 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:06.719363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-173f0d28-170e-4543-addc-00aae600d26b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.099253 May 23 13:00:06.722208 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 9/23] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:00:05 2022] POST /compute/v2.1/servers => generated 383 bytes in 1103 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:00:06.757526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1cfd135f-8a2a-4739-92fb-6f34e6d57f4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:06.759200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1cfd135f-8a2a-4739-92fb-6f34e6d57f4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:07.013976 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1cfd135f-8a2a-4739-92fb-6f34e6d57f4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:07.029391 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1cfd135f-8a2a-4739-92fb-6f34e6d57f4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1221 microversion: 2.1 time: 0.296507 May 23 13:00:07.047912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 15/24] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:06 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1221 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:07.268042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-891390768", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:07.374434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] validate_networks() for [('11cfb087-2166-41ce-9a36-a5b349287bd2', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:00:07.536710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:07.537902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:07.555398 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:07.557249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:07.560174 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:07.560841 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:07.731770 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:07.747503 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-151b071e-0174-4370-85fb-29a137a102d2 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1218 microversion: 2.1 time: 0.214586 May 23 13:00:07.753319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 16/25] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:07 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1218 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:07.762180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a11a7704-75d0-4e5c-8c66-abf9a8d7c8df tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b183628b0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:07.819633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a11a7704-75d0-4e5c-8c66-abf9a8d7c8df tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 287 microversion: 2.1 time: 0.064791 May 23 13:00:07.820354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 17/26] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:07 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 287 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:07.832627 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4aabaf0b-2c34-4aa3-94d1-e68ba86d2e55 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18bbe940>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:08.040373 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4aabaf0b-2c34-4aa3-94d1-e68ba86d2e55 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "DELETE /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 202 len: 0 microversion: 2.1 time: 0.212024 May 23 13:00:08.041723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 18/27] 10.222.0.30 () {58 vars in 1253 bytes} [Mon May 23 13:00:07 2022] DELETE /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 0 bytes in 215 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:00:08.051764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c9286892-d1e5-4c67-834c-b7b9a02436d2 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18bbe9d0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:08.101170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c9286892-d1e5-4c67-834c-b7b9a02436d2 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.053180 May 23 13:00:08.101984 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 19/28] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:08 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:00:08.328459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:00:08.337795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:00:08.341621 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:00:08.342274 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:00:08.343776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:00:08.389445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:08.392582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:08.395730 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:08.399994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:08.434680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:00:08.439981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for user b18e5f1b24634096995068e41d94cad6 and project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:00:08.460055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:00:08.462516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:00:08.623323 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a2ae6b05-005c-4ba0-8af8-072dd8ae3eb8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.359170 May 23 13:00:08.625796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 10/29] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:00:07 2022] POST /compute/v2.1/servers => generated 383 bytes in 1366 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:00:08.641257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-db5503ec-62a0-4ab9-8ecd-c7c62d0b9750 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:08.643294 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-db5503ec-62a0-4ab9-8ecd-c7c62d0b9750 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:08.774306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:08.775636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:08.810777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:08.811454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:08.822822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:08.824171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:08.841473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-db5503ec-62a0-4ab9-8ecd-c7c62d0b9750 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:08.849570 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-db5503ec-62a0-4ab9-8ecd-c7c62d0b9750 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1221 microversion: 2.1 time: 0.213729 May 23 13:00:08.856087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 20/30] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:08 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1221 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:08.943715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:08.962562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9c22775a-991b-4e92-b2d7-92ca5569015d tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1266 microversion: 2.1 time: 0.195316 May 23 13:00:08.963608 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 11/31] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:08 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1266 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:09.114456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-06facbcd-8ab1-4db9-bfed-d7a13530463b tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b183628b0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:09.158117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-06facbcd-8ab1-4db9-bfed-d7a13530463b tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.047341 May 23 13:00:09.158609 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 21/32] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:09 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:09.181605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1095450753", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "27240b79-2fbf-4ada-bcf2-ef920669591c"}], "security_groups": [{"name": "tempest-securitygroup--753749126"}], "key_name": "tempest-keypair-756628403", "user_data": "}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:09.337541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] validate_networks() for [('27240b79-2fbf-4ada-bcf2-ef920669591c', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:00:09.871723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:09.871723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:09.880509 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:09.881147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:09.882121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:09.883625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:09.976638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:00:09.977329 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:00:09.977964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:00:09.978498 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:00:09.979184 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:00:09.994610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:09.995055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:09.995919 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:10.002366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:10.039409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:00:10.044442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:00:10.050096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:10.058816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:00:10.061385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:00:10.066797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-77e11d97-5359-4627-990b-1404591ade40 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1225 microversion: 2.1 time: 0.202368 May 23 13:00:10.068356 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 22/33] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:09 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1225 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:10.074957 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:10.075924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:10.084522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:10.084522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:10.084795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:10.085293 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:10.207145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3e5d0b94-0f0a-426d-aa7f-1b9474110970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.033143 May 23 13:00:10.215711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 12/34] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:00:09 2022] POST /compute/v2.1/servers => generated 401 bytes in 1042 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:00:10.231679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:10.239992 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-67940bf8-7b99-4178-b3bc-32e581f9aabb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1266 microversion: 2.1 time: 0.170422 May 23 13:00:10.243016 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 23/35] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:10 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:10.244901 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-16edbc5d-05cd-4bb0-b562-47cd3f69897b tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18343af0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:10.257486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-29835f41-452d-493c-83d8-408bd57b5714 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:10.258333 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-29835f41-452d-493c-83d8-408bd57b5714 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:10.305942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-16edbc5d-05cd-4bb0-b562-47cd3f69897b tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.082757 May 23 13:00:10.310300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 13/36] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:10 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:10.370691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-29835f41-452d-493c-83d8-408bd57b5714 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:10.380515 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-29835f41-452d-493c-83d8-408bd57b5714 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1241 microversion: 2.1 time: 0.134183 May 23 13:00:10.391289 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 24/37] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:10 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1241 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:11.086561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:11.087235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:11.095739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:11.096199 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:11.096974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:11.097412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:11.161980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:11.170590 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-63b059b5-69c6-4883-ae9d-3ca9f48111bf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1225 microversion: 2.1 time: 0.089455 May 23 13:00:11.171174 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 14/38] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:11 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:11.259491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:11.260103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:11.271581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:11.277949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:11.279317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:11.280087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:11.318659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8486b1b4-0947-4ca6-a269-e3a48f11af87 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18326f70>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:11.350604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:11.359477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8486b1b4-0947-4ca6-a269-e3a48f11af87 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.041525 May 23 13:00:11.359477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 15/39] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:11 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:00:11.359617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3ec233ff-691b-47aa-9347-70988b70f4ad tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1266 microversion: 2.1 time: 0.104322 May 23 13:00:11.359617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 25/40] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:11 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1266 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:11.399677 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:11.400336 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:11.407474 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:11.407927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:11.408747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:11.409188 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:11.479186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:11.486951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3cee382d-8dd9-41da-a4a7-bc93b7947156 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1245 microversion: 2.1 time: 0.091141 May 23 13:00:11.487523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 16/41] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:11 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1245 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:12.190693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:12.191934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:12.200966 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:12.202578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:12.203578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:12.204189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:12.333843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:12.346510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-bd396c18-68ed-4cf8-a21f-645414451e49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1273 microversion: 2.1 time: 0.160082 May 23 13:00:12.347365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 26/42] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:12 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1273 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:12.367078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1211913f-02d3-4dd6-a577-8aab3855c23e tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b182d8b80>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:12.377766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:12.378427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:12.385097 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:12.385573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:12.387098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:12.387098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:12.410971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1211913f-02d3-4dd6-a577-8aab3855c23e tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 200 len: 286 microversion: 2.1 time: 0.047952 May 23 13:00:12.412163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 17/43] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:12 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 286 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:12.455839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:12.464848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-88de4927-06ad-4a1c-9571-5f38d39be416 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1272 microversion: 2.1 time: 0.092433 May 23 13:00:12.466065 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 27/44] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:12 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1272 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:12.504746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:12.505515 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:12.515204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:12.516543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:12.518206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:12.518987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:12.625121 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:12.635356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ffe1b53e-3dd4-485a-8a52-b77b84e8d938 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1293 microversion: 2.1 time: 0.134935 May 23 13:00:12.637714 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 18/45] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:12 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1293 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:13.365935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:13.367477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:13.394289 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:13.395050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:13.396928 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:13.399308 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:13.424575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cf426749-300f-447d-818f-003486d8395a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18326ca0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:13.674232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:13.694854 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b7b5e20c-31a7-45e5-a700-3b3c1f1ed976 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1291 microversion: 2.1 time: 0.333712 May 23 13:00:13.710146 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 28/46] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:13 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1291 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:13.738620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:13.740893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:13.759567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:13.762737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:13.764292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:13.765979 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:14.295516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-cf426749-300f-447d-818f-003486d8395a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] HTTP exception thrown: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cf426749-300f-447d-818f-003486d8395a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Request method failure captured: May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d HTTP/1.1 May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VJ0WxNt6pWGMVYZD7f5A6saFfFsX4EOrJhKENVibysYpihgj2LxOHO7lxegnAbaWJLHRR99QqmT-bqaSg2O9EmW2-hrJXLNsvVesQca_wbk-RtjssqgAzUPcc--UKiG1G9qmqOzX9SWyraI2_VfqEfkc7BUTdeuNiNDgPBtihzMII May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:00:14.299630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: caaf63946cde4042b60389be276025d0 May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '780558d5-fdbe-411b-b8dd-aaf3c80f514d'} May 23 13:00:14.300674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. (HTTP 404) (Request-ID: req-adb586e1-76a0-46ae-bc6e-857b9e3cb8c0) May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.301366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:14.302318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show May 23 13:00:14.303385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cf426749-300f-447d-818f-003486d8395a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Returning 404 to user: Snapshot 780558d5-fdbe-411b-b8dd-aaf3c80f514d could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:00:14.304374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cf426749-300f-447d-818f-003486d8395a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d" status: 404 len: 111 microversion: 2.1 time: 0.880818 May 23 13:00:14.314326 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 19/47] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:00:13 2022] GET /compute/v2.1/os-snapshots/780558d5-fdbe-411b-b8dd-aaf3c80f514d => generated 111 bytes in 890 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:00:14.315386 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:14.316005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:14.328617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:14.345105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d69d001f-b30f-41b5-a9d8-4dd69f2429cb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.631204 May 23 13:00:14.358822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:14.358822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:14.359665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 29/48] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:13 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:14.360013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:14.360013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:14.370725 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d44f169c-db2b-4249-a92c-c800a8033811 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b18371ee0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:14.462849 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d44f169c-db2b-4249-a92c-c800a8033811 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/ef8c2c89-18e5-451a-9536-33904a04d621" status: 200 len: 350 microversion: 2.1 time: 0.114395 May 23 13:00:14.463602 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 30/49] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:14 2022] GET /compute/v2.1/os-volumes/ef8c2c89-18e5-451a-9536-33904a04d621 => generated 350 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:14.473848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c2796e8b-d3e7-43a4-991f-a76336579ba1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18362f70>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:14.584123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:14.599800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a3449deb-b856-47b9-b889-99f166fc3e2d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1299 microversion: 2.1 time: 0.288859 May 23 13:00:14.621011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 20/50] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:14 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1299 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:14.632476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fcdfe78e-e216-4057-bbc2-23e77b0f2c38 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b182dcee0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:14.765617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fcdfe78e-e216-4057-bbc2-23e77b0f2c38 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.141960 May 23 13:00:14.766213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 21/51] 10.222.0.30 () {58 vars in 1133 bytes} [Mon May 23 13:00:14 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:14.773314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:14.779446 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:14.802120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:14.802603 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:14.803429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:14.803872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:14.896916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c2796e8b-d3e7-43a4-991f-a76336579ba1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "DELETE /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 202 len: 0 microversion: 2.1 time: 0.428500 May 23 13:00:14.897616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 31/52] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:00:14 2022] DELETE /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 0 bytes in 430 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:00:14.909395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-44d93a43-42af-48e2-bbef-5b015bdb8ca4 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b183113a0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:14.923577 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:14.935608 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-97313baa-5c27-4eb9-83fd-1269e9438f44 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1279 microversion: 2.1 time: 0.167465 May 23 13:00:14.937227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 22/53] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:14 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1279 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:14.943152 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-db41632b-4e1b-44a9-9025-b235a70c7a3e tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b1830ba60>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.036160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-db41632b-4e1b-44a9-9025-b235a70c7a3e tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.095491 May 23 13:00:15.039525 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 23/54] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:14 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:15.088443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-44d93a43-42af-48e2-bbef-5b015bdb8ca4 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.184062 May 23 13:00:15.089742 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 32/55] 10.222.0.30 () {58 vars in 1148 bytes} [Mon May 23 13:00:14 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:15.110031 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d7fc817d-3632-4f13-8f70-ea52394cae3d tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b1830bdc0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.234569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d7fc817d-3632-4f13-8f70-ea52394cae3d tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.136388 May 23 13:00:15.235724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 24/56] 10.222.0.30 () {58 vars in 1133 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:15.248100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3c30c309-7efd-4d29-a94a-b97d2176439c tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b18362790>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.376864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.377518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:15.391389 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:15.391864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:15.396135 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:15.398988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:15.432763 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3c30c309-7efd-4d29-a94a-b97d2176439c tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.189532 May 23 13:00:15.440581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 33/57] 10.222.0.30 () {58 vars in 1166 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:15.445676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-50b6f864-39fb-45e0-9c68-d5978bd18509 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b18358940>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.786186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-50b6f864-39fb-45e0-9c68-d5978bd18509 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.197545 May 23 13:00:15.786186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 34/58] 10.222.0.30 () {58 vars in 1169 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:15.786186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.786186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:15.786186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:15.787161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:15.787161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:15.787161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:15.787555 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:15.787555 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4129ea98-cea5-47b4-b578-68e3077fb7ab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1299 microversion: 2.1 time: 0.123989 May 23 13:00:15.787555 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 35/59] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1299 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:15.787800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:15.787800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2d6b5fe8-b3dc-4a9f-a8f0-e088ec0ddc75 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.411330 May 23 13:00:15.787800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 25/60] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:15.815436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-54d8ac66-8188-42ec-8f92-8286d441e6c7 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b196ae280>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.906395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-54d8ac66-8188-42ec-8f92-8286d441e6c7 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.099513 May 23 13:00:15.910733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 36/61] 10.222.0.30 () {58 vars in 1154 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:15.915866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c05adcdc-9659-41f0-b40c-ecf4eeaebce8 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b1830b0d0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.955665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:15.956291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:15.972072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:15.974814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:15.975640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:15.976770 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:16.028945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c05adcdc-9659-41f0-b40c-ecf4eeaebce8 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.117222 May 23 13:00:16.029695 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 26/62] 10.222.0.30 () {58 vars in 1187 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:16.041169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7d1f3c38-4a52-4b34-9f2c-0939a9538cb6 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] Calling method '.version_select at 0x7f9b182d8790>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:16.051035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:16.060276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6b3cfd73-b77a-46c9-a8ee-10fb663ef5bb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1279 microversion: 2.1 time: 0.110970 May 23 13:00:16.061082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 37/63] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:15 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1279 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:16.065987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8a097f18-ccef-4826-8930-ea44df9322f6 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b196ae0d0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:16.131567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8a097f18-ccef-4826-8930-ea44df9322f6 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.069122 May 23 13:00:16.136745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 38/64] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:16 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:16.161568 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7d1f3c38-4a52-4b34-9f2c-0939a9538cb6 tempest-VolumesTestJSON-1550337676 tempest-VolumesTestJSON-1550337676-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.124109 May 23 13:00:16.175662 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 27/65] 10.222.0.30 () {58 vars in 1154 bytes} [Mon May 23 13:00:16 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:16.810858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:16.811588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:16.811996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:16.812191 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:16.818205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:16.818659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:16.819431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:16.819860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:16.820157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:16.821044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:16.822285 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:16.823891 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:17.014394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:17.022368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c17a2e64-6867-4251-a178-cef603a94085 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.216329 May 23 13:00:17.023674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 39/66] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:16 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:17.024808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:17.034868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-95f7890f-a03f-416a-9737-5e19df947f4b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.228718 May 23 13:00:17.042833 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 28/67] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:16 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:17.077289 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:17.077833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:17.083772 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:17.084226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:17.084994 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:17.085440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:17.143939 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-630828e1-a632-4288-8972-0d52fc20e77a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b1830b5e0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:17.203644 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-630828e1-a632-4288-8972-0d52fc20e77a tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.063478 May 23 13:00:17.204325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 29/68] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:17 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:17.226982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:17.234710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1fbc5a7b-f347-402a-b512-dcf96fc343e0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.161595 May 23 13:00:17.235270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 40/69] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:17 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:18.039957 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:18.040579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:18.047314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:18.047777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:18.048547 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:18.048984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:18.052338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:18.052952 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:18.060135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:18.060610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:18.061386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:18.061823 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:18.258369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:18.263000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:18.267089 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bef77f43-1969-4df1-a32f-1a7983ab6729 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.230930 May 23 13:00:18.267748 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 30/70] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:18 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:18.272260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1714d34f-c1df-4ca6-a638-768fbf1b0fda tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b182d8c10>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:18.275498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-773b7a91-9cbe-4706-961c-33bb414acf98 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.226818 May 23 13:00:18.293011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 41/71] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:18 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:18.293011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:18.293011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:18.293011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:18.293324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:18.293324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:18.293619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:18.321374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1714d34f-c1df-4ca6-a638-768fbf1b0fda tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.052553 May 23 13:00:18.321950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 31/72] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:18 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:18.426302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:18.434121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ee6aa930-9091-45d9-bec8-c34859e28b8a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.153520 May 23 13:00:18.434687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 42/73] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:18 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:19.284458 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:19.285062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:19.291279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:19.292446 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:19.293400 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:19.294007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:19.299873 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:19.300476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:19.310698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:19.311137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:19.311897 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:19.312322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:19.332485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:19.534643 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:19.539146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:19.542423 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-53848285-28ba-42f4-9c98-c7aa9f9629eb tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.246117 May 23 13:00:19.543018 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 43/74] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:19 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:19.547656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cb443b8f-3c9b-422a-af74-de266e2761b7 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18358a60>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:19.547893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-504a5e0a-1ae9-4784-84a1-bed7f7d0185a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.267083 May 23 13:00:19.548457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 32/75] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:19 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:19.553160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:19.554061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:19.560377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:19.560796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:19.561454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:19.561896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:19.595711 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cb443b8f-3c9b-422a-af74-de266e2761b7 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.051546 May 23 13:00:19.596290 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 44/76] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:19 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:19.707492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:19.715642 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-93bd2b30-8e0d-475f-adb2-c33bfe87c819 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.166008 May 23 13:00:19.716207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 33/77] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:19 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:20.556099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=12 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:20.561483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:20.562422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:20.566066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:20.571682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:20.571803 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:20.571803 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:20.571988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=8 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:20.572078 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:20.572161 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=28 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:20.572249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:20.572338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=12 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:20.576335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:20.577103 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:20.578472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:20.579096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:20.804323 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:20.816946 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-049259f4-5295-48f1-8935-0895beffc41c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.254924 May 23 13:00:20.826443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 34/78] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:20 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:20.833202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b7286bb5-9ece-484e-b3bf-784b75b29a74 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b182d8af0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:20.850792 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:20.865830 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2adb2c26-6d51-4db3-b104-04473dd189f8 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.310283 May 23 13:00:20.877504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 45/79] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:20 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:20.879477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:20.880196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:20.893330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:20.894014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:20.895317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:20.895869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:20.937256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b7286bb5-9ece-484e-b3bf-784b75b29a74 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 200 len: 345 microversion: 2.1 time: 0.109338 May 23 13:00:20.943959 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 35/80] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:20 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 345 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:21.060975 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:21.080456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-345469ef-2f38-4c70-a5cf-6bfc9a4e9460 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.204353 May 23 13:00:21.081932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 46/81] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:20 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:21.839913 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:21.840511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:21.850186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:21.850639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:21.851412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:21.853539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:21.889431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:21.890761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:21.898631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:21.899792 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:21.900843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:21.901519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:22.180792 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:22.194623 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-72189c36-716e-4ab8-b348-9130374e9769 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.309455 May 23 13:00:22.212618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 47/82] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:21 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:22.212618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ec137716-a827-4c2e-9aa7-9978b2349dac tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18bbe9d0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:22.213224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:22.223370 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-056561f6-1207-4346-a02b-dc7b614c08d5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.388028 May 23 13:00:22.223959 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 36/83] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:21 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:22.235404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:22.241224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:22.247657 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ec137716-a827-4c2e-9aa7-9978b2349dac tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] HTTP exception thrown: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.250224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:22.250768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:22.251558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:22.252057 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ec137716-a827-4c2e-9aa7-9978b2349dac tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Request method failure captured: May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 HTTP/1.1 May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VJ0WxNt6pWGMVYZD7f5A6saFfFsX4EOrJhKENVibysYpihgj2LxOHO7lxegnAbaWJLHRR99QqmT-bqaSg2O9EmW2-hrJXLNsvVesQca_wbk-RtjssqgAzUPcc--UKiG1G9qmqOzX9SWyraI2_VfqEfkc7BUTdeuNiNDgPBtihzMII May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:00:22.252454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: caaf63946cde4042b60389be276025d0 May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '362e9bb0-0628-436a-b4cd-2711c6bf4cc4'} May 23 13:00:22.254154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. (HTTP 404) (Request-ID: req-7520f804-068d-481d-b183-babb36b197aa) May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.255090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:22.256318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 13:00:22.260851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ec137716-a827-4c2e-9aa7-9978b2349dac tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Returning 404 to user: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ec137716-a827-4c2e-9aa7-9978b2349dac tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 404 len: 109 microversion: 2.1 time: 0.061939 May 23 13:00:22.262133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 48/84] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:22 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 109 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:00:22.272415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-01b029c3-50ff-410d-a9b2-0d50b56deec9 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18d33940>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:22.307331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-01b029c3-50ff-410d-a9b2-0d50b56deec9 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] HTTP exception thrown: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-01b029c3-50ff-410d-a9b2-0d50b56deec9 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Request method failure captured: May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: DELETE /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 HTTP/1.1 May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VJ0WxNt6pWGMVYZD7f5A6saFfFsX4EOrJhKENVibysYpihgj2LxOHO7lxegnAbaWJLHRR99QqmT-bqaSg2O9EmW2-hrJXLNsvVesQca_wbk-RtjssqgAzUPcc--UKiG1G9qmqOzX9SWyraI2_VfqEfkc7BUTdeuNiNDgPBtihzMII May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:00:22.310193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: caaf63946cde4042b60389be276025d0 May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '362e9bb0-0628-436a-b4cd-2711c6bf4cc4'} May 23 13:00:22.311621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. (HTTP 404) (Request-ID: req-f84a8040-e3de-4551-bd0a-6b0c8cae08f3) May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.312453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 13:00:22.313755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 23 13:00:22.315229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-01b029c3-50ff-410d-a9b2-0d50b56deec9 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Returning 404 to user: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-01b029c3-50ff-410d-a9b2-0d50b56deec9 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "DELETE /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 404 len: 109 microversion: 2.1 time: 0.044507 May 23 13:00:22.316406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 49/85] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:00:22 2022] DELETE /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 109 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:00:22.323634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa69c8a5-f616-4134-b97c-1cada34a98d1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Calling method '.version_select at 0x7f9b18bbedc0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:22.352437 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-fa69c8a5-f616-4134-b97c-1cada34a98d1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] HTTP exception thrown: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa69c8a5-f616-4134-b97c-1cada34a98d1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Request method failure captured: May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 HTTP/1.1 May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VJ0WxNt6pWGMVYZD7f5A6saFfFsX4EOrJhKENVibysYpihgj2LxOHO7lxegnAbaWJLHRR99QqmT-bqaSg2O9EmW2-hrJXLNsvVesQca_wbk-RtjssqgAzUPcc--UKiG1G9qmqOzX9SWyraI2_VfqEfkc7BUTdeuNiNDgPBtihzMII May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:00:22.354971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e6ff065adc71404184a1fe9ccdf97bee"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: e6ff065adc71404184a1fe9ccdf97bee May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-156821474 May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: caaf63946cde4042b60389be276025d0 May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesSnapshotsTestJSON-156821474-project May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '362e9bb0-0628-436a-b4cd-2711c6bf4cc4'} May 23 13:00:22.356106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. (HTTP 404) (Request-ID: req-3165e1ad-9193-437a-a88c-e9a1bc73c1b2) May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.356782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:22.357710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 13:00:22.358755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa69c8a5-f616-4134-b97c-1cada34a98d1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] Returning 404 to user: Volume 362e9bb0-0628-436a-b4cd-2711c6bf4cc4 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fa69c8a5-f616-4134-b97c-1cada34a98d1 tempest-VolumesSnapshotsTestJSON-156821474 tempest-VolumesSnapshotsTestJSON-156821474-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4" status: 404 len: 109 microversion: 2.1 time: 0.037703 May 23 13:00:22.359716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 50/86] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:22 2022] GET /compute/v2.1/os-volumes/362e9bb0-0628-436a-b4cd-2711c6bf4cc4 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:00:22.428913 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:22.448154 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4a2eacc0-910e-49fe-b6ea-0c39bf1c6137 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.222362 May 23 13:00:22.448916 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 37/87] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:22 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:23.223014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:23.223623 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:23.232728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:23.233387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:23.234656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:23.235488 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:23.248220 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:23.248220 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:23.268180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:23.268631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:23.269422 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:23.271443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:23.282027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:24.017589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:24.039568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-383e9d01-21e5-4830-ab15-d4b61c1cb8e4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.821623 May 23 13:00:24.043695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 51/88] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:23 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 827 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:24.059687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:24.060456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:24.072146 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:24.076638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:24.079269 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:24.080444 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:24.381225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:24.399374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1304378c-cc6b-4746-99de-1a611be24fba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 1.162470 May 23 13:00:24.402227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 38/89] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:23 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 1166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:24.899926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:24.909224 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0d7813da-a0a3-45a5-bdff-f04c2b559096 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.863611 May 23 13:00:24.912207 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 52/90] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:24 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:25.063866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:25.066397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:25.087712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.088454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.089298 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.089738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.581211 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-9eb95ace-1c1d-4523-b44c-f06148e21a95 req-f1783768-9384-44cd-aefb-84523536f88e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbf0c0b3-5059-405a-b847-07ed7c6edee9", "tag": "65ac147c-5bdb-47fa-adf1-5827d23a6e48"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:25.588798 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:25.596713 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-9eb95ace-1c1d-4523-b44c-f06148e21a95 req-f1783768-9384-44cd-aefb-84523536f88e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.597500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-9eb95ace-1c1d-4523-b44c-f06148e21a95 req-f1783768-9384-44cd-aefb-84523536f88e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.628837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-9eb95ace-1c1d-4523-b44c-f06148e21a95 req-f1783768-9384-44cd-aefb-84523536f88e service nova] Creating event network-changed:65ac147c-5bdb-47fa-adf1-5827d23a6e48 for instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 on hv-cinder-maste May 23 13:00:25.630292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-9eb95ace-1c1d-4523-b44c-f06148e21a95 req-f1783768-9384-44cd-aefb-84523536f88e service nova] Acquired lock "compute-rpcapi-router" {{(pid=130046) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 23 13:00:25.632671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.644292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.645907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.646530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.660037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.compute.rpcapi [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 23 13:00:25.661382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] Releasing lock "compute-rpcapi-router" {{(pid=130046) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 23 13:00:25.690913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-820ce2f1-5c69-499e-9d72-0f4c047cfbaa None None] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.488949 May 23 13:00:25.695920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 53/91] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:25.709428 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:25.710567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:25.728192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.730462 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.732320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.740593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.782392 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:25.815246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4b3aea05-65cb-4bbe-a146-38c18801f689 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.753037 May 23 13:00:25.818508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 39/92] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:25 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:25.931389 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:25.936946 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:25.953691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.954187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:25.960263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:25.960938 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:26.799404 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:26.818873 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ec8c9c23-57c0-4a17-8c25-b08d464b19a4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 1.121996 May 23 13:00:26.830524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 54/93] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:25 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 1134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:26.859697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:26.860458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:26.880775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:26.885632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:26.885632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:26.885933 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:27.320395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:27.336671 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-648b452e-b122-4fe6-8f0b-b9cc57abd481 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 1.406417 May 23 13:00:27.336671 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 40/94] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:25 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 1411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:27.734215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:27.734215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6dd344d2-4690-43ed-8dbc-0d2ee0d465c6 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.848310 May 23 13:00:27.734215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 55/95] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:26 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:27.840293 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:27.855635 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:27.887091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:27.887543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:27.890179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:27.890848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:28.355381 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:28.355950 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:28.367209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:28.367962 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:28.372082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:28.372789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:29.039703 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:29.066739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e31bb931-e4b6-485b-986c-6fe9f215dd0b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 1.218204 May 23 13:00:29.066739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 41/96] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:27 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 1223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:29.071721 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:29.074743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:29.087579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:29.088032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:29.088839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:29.089266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:29.145930 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:29.161191 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3ad48236-643f-41ce-bf11-a75c37457c49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.807621 May 23 13:00:29.162250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 56/97] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:28 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:29.303936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:29.312004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-277c81db-034c-4f60-b22a-7f4bbf3a433a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.253532 May 23 13:00:29.312575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 42/98] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:29 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:29.537082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-bced82df-6f5e-4db7-86ac-b4f817fcf143 req-f066a425-5f98-4a67-af05-f9e70661aec2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e01afbdf-8ced-4f91-b14c-b87b2fcf1506", "tag": "97ab468b-d2e1-4b56-98d6-23dfeb309dc3"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:29.549797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-bced82df-6f5e-4db7-86ac-b4f817fcf143 req-f066a425-5f98-4a67-af05-f9e70661aec2 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:29.550282 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-bced82df-6f5e-4db7-86ac-b4f817fcf143 req-f066a425-5f98-4a67-af05-f9e70661aec2 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:29.561816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-bced82df-6f5e-4db7-86ac-b4f817fcf143 req-f066a425-5f98-4a67-af05-f9e70661aec2 service nova] Creating event network-changed:97ab468b-d2e1-4b56-98d6-23dfeb309dc3 for instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 on hv-cinder-maste May 23 13:00:29.567491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-bced82df-6f5e-4db7-86ac-b4f817fcf143 req-f066a425-5f98-4a67-af05-f9e70661aec2 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.403525 May 23 13:00:29.568084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 57/99] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:30.074142 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:30.074749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:30.080719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:30.081158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:30.082173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:30.082536 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:30.178778 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:30.179760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:30.188403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:30.189059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:30.190060 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:30.194981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:30.229808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:30.237898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9e0270e4-1204-48c3-87ed-c7ae4f8c63a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.167746 May 23 13:00:30.239107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 43/100] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:30 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:30.329680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:30.330608 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:30.339356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:30.341267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:30.344436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:30.346216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:30.394177 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:30.404438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-27b77b0f-7013-4e89-8ab1-ff168e39f9c5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.230145 May 23 13:00:30.413606 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 58/101] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:30 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:30.538215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:30.550679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bb5c2f2e-f476-4336-bcc3-6e97094e7f6c tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.225331 May 23 13:00:30.554789 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 44/102] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:30 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:31.255607 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:31.256613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:31.271261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.271829 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.272701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.274547 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.289330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a668f236-f212-4375-8d14-c9ff3074487a tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Action: 'create', calling method: .version_select at 0x7f9b18311160>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1002599414", "metadata": {"Type": "work"}}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:31.484402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:31.493222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6b67eab7-2d09-4f9f-83c4-c3ca35f2ee87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.241437 May 23 13:00:31.494429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 45/103] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:31 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:31.507852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:31.508469 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:31.515040 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.515714 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.516491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.516946 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.656901 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:31.664252 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5ec1607c-7eac-4aff-a2b4-8462e6698431 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.160089 May 23 13:00:31.664818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 46/104] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:31 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:31.669838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:31.670466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:31.676083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.676524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.677272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.677699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.828394 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:31.836288 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-69d187b9-797a-48ef-bf98-38c2d643d3ac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.170502 May 23 13:00:31.836856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 47/105] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:31 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:31.846395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-78fdd2d2-b084-468c-8eb3-dbcf301bb071 req-dc58a48e-8071-4ab8-a188-3586da524bd8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "825e6db1-f2ed-47d4-88dc-07191da24f5f", "tag": "b2757ca5-c976-47ae-9a5d-46786a7e1473"}, {"name": "network-changed", "server_uuid": "3d4562fc-a49b-4d0d-b486-e8d2fb59db7c", "tag": "381aa96b-0ff9-4824-b7af-8578a2b139b2"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:31.859784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-78fdd2d2-b084-468c-8eb3-dbcf301bb071 req-dc58a48e-8071-4ab8-a188-3586da524bd8 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.861478 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-78fdd2d2-b084-468c-8eb3-dbcf301bb071 req-dc58a48e-8071-4ab8-a188-3586da524bd8 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.882722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-78fdd2d2-b084-468c-8eb3-dbcf301bb071 req-dc58a48e-8071-4ab8-a188-3586da524bd8 service nova] Creating event network-changed:b2757ca5-c976-47ae-9a5d-46786a7e1473 for instance 825e6db1-f2ed-47d4-88dc-07191da24f5f on hv-cinder-maste May 23 13:00:31.883223 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-78fdd2d2-b084-468c-8eb3-dbcf301bb071 req-dc58a48e-8071-4ab8-a188-3586da524bd8 service nova] Creating event network-changed:381aa96b-0ff9-4824-b7af-8578a2b139b2 for instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c on hv-cinder-maste May 23 13:00:31.883749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-78fdd2d2-b084-468c-8eb3-dbcf301bb071 req-dc58a48e-8071-4ab8-a188-3586da524bd8 service nova] Acquired lock "compute-rpcapi-router" {{(pid=130045) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 23 13:00:31.885163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.885633 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.886434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:31.889394 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:31.895961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.compute.rpcapi [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 23 13:00:31.896818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] Releasing lock "compute-rpcapi-router" {{(pid=130045) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 23 13:00:31.913330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3fe65844-f1db-4b18-b4c3-a28b68272ff0 None None] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.074620 May 23 13:00:31.913940 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 48/106] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:31 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:32.342632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a668f236-f212-4375-8d14-c9ff3074487a tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 1.515901 May 23 13:00:32.343256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 59/107] 10.222.0.30 () {60 vars in 1155 bytes} [Mon May 23 13:00:30 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 1517 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 23 13:00:32.357996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dc2d2b1a-84df-404c-9999-034ec067cfac tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b18270b80>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:32.414794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dc2d2b1a-84df-404c-9999-034ec067cfac tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 200 len: 353 microversion: 2.1 time: 0.063491 May 23 13:00:32.420724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 49/108] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:32 2022] GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 353 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:32.510831 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:32.511438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:32.518442 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:32.518887 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:32.519654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:32.520086 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:32.674773 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:32.684812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:32.685471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6208f10c-2692-4fa7-8748-67c106b476da tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.178157 May 23 13:00:32.686551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 60/109] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:32 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:32.686726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:32.702234 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:32.703615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:32.704637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:32.705182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:32.854989 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:32.855632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:32.864023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:32.864186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:32.864186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:32.864186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:32.864618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:32.868438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bce7aaf8-8f90-4ca2-8bf5-3ee2febf4640 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.190168 May 23 13:00:32.869426 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 50/110] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:32 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:33.001758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:33.009739 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c2d342ce-9a0d-4b3f-93c1-a301d4a5b84f tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.158746 May 23 13:00:33.010571 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 61/111] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:32 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:33.432308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f316dbf9-82fb-4515-a840-4fe270a2c36a tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b1830b430>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:33.485575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f316dbf9-82fb-4515-a840-4fe270a2c36a tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 200 len: 353 microversion: 2.1 time: 0.057630 May 23 13:00:33.486185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 51/112] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:33 2022] GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 353 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:00:33.702991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:33.703595 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:33.709595 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:33.710085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:33.713405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:33.713405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:33.860302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:33.868681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-016e9dd4-a23f-45ea-b601-42c11524f1cc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.169985 May 23 13:00:33.869514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 62/113] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:33 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:33.892491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:33.893010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:33.899702 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:33.900157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:33.900926 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:33.901932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:34.031052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:34.037060 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:34.044476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:34.046415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:34.047417 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:34.048005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:34.447519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:34.470609 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d8891e06-1379-41c6-b002-95ae78ad3784 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.571070 May 23 13:00:34.473270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 52/114] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:33 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:34.498954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fe57eb45-8bd0-4d16-97c7-c9e4f3ec69ab tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b1830bb80>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:34.539874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:34.551725 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8ab8932b-b673-4fed-b184-312e611e6b92 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.528868 May 23 13:00:34.561922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 63/115] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:34 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:34.562515 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fe57eb45-8bd0-4d16-97c7-c9e4f3ec69ab tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 200 len: 354 microversion: 2.1 time: 0.067676 May 23 13:00:34.562870 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 53/116] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:34 2022] GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 354 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:34.580956 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d715b241-941b-49b4-8df8-74479ef72feb tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b18311280>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:34.639381 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d715b241-941b-49b4-8df8-74479ef72feb tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 200 len: 354 microversion: 2.1 time: 0.064053 May 23 13:00:34.640721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 64/117] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:34 2022] GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 354 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:34.657683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-801904f6-eb81-4674-9a49-73be44c79e2c tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b1830b5e0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:34.884544 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:34.885155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:34.894520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:34.894971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:34.895736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:34.896174 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:34.989547 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-801904f6-eb81-4674-9a49-73be44c79e2c tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "DELETE /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 202 len: 0 microversion: 2.1 time: 0.333599 May 23 13:00:34.989547 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 54/118] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:00:34 2022] DELETE /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 0 bytes in 335 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:00:34.996907 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ae020b4f-c25d-4e37-b5a2-b9e40a3221c4 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b196ab280>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:35.098486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:35.119027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6baf62f0-6529-4b23-b0e4-8fe482d0f54d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.238263 May 23 13:00:35.119616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 65/119] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:34 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:35.195823 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ae020b4f-c25d-4e37-b5a2-b9e40a3221c4 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 200 len: 353 microversion: 2.1 time: 0.202858 May 23 13:00:35.196905 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 55/120] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:34 2022] GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 353 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:35.487677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:35.488219 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:35.494285 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:35.494766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:35.495592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:35.496081 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:35.575366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:35.577634 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:35.586785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:35.587832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:35.589256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:35.590043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:35.659060 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:35.696367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:35.707718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f4f4d55e-5d7e-4606-96f1-6b19746c05d9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.224005 May 23 13:00:35.708292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 66/121] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:35 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:35.772654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:35.781808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0d36dfb0-05c3-4326-9e53-e6c5bd048dd4 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.214072 May 23 13:00:35.782595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 56/122] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:35 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:36.136475 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:36.137105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:36.143753 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:36.144195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:36.144951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:36.145437 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:36.150034 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:36.208902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6e9dd0a-71d2-4a08-9738-b9fa2d6312d5 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Calling method '.version_select at 0x7f9b182d5670>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:36.248524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-a6e9dd0a-71d2-4a08-9738-b9fa2d6312d5 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] HTTP exception thrown: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6e9dd0a-71d2-4a08-9738-b9fa2d6312d5 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Request method failure captured: May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 HTTP/1.1 May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4Vupnnc8lQ-e8UQVmSnTiAPqEB8ld4rerFi9axzhgw-dQQ2CFs9bqxwGUe9fuiPhMXgLPeRd1NR3NC4oQStmAH1gEvT2wIJs4eRtEkBnsBgmokf2XJyNRZMQATHkSJK3JA-Jfka92MIl_KhJeOJSEp00VYYvGlJ4g5c8DN0Nqegz40 May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: de4eed939156431fa99178951d5fcbec May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesGetTestJSON-1967928423 May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:00:36.251001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_de4eed939156431fa99178951d5fcbec"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/de4eed939156431fa99178951d5fcbec"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/de4eed939156431fa99178951d5fcbec"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/de4eed939156431fa99178951d5fcbec"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesGetTestJSON-1967928423 May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: de4eed939156431fa99178951d5fcbec May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesGetTestJSON-1967928423 May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesGetTestJSON-1967928423-project May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 7adac7fe4243402aa25085cacfe2b29d May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesGetTestJSON-1967928423-project May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'bd8190cc-fa6f-4a97-bf3c-4952bf0e8145'} May 23 13:00:36.252448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. (HTTP 404) (Request-ID: req-240971d0-dcde-4e2f-8c54-c198ac61a247) May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:36.253313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:00:36.254438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 13:00:36.255624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. May 23 13:00:36.256669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:00:36.258924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6e9dd0a-71d2-4a08-9738-b9fa2d6312d5 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] Returning 404 to user: Volume bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:00:36.259972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a6e9dd0a-71d2-4a08-9738-b9fa2d6312d5 tempest-VolumesGetTestJSON-1967928423 tempest-VolumesGetTestJSON-1967928423-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145" status: 404 len: 109 microversion: 2.1 time: 0.054978 May 23 13:00:36.261692 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 57/123] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:00:36 2022] GET /compute/v2.1/os-volumes/bd8190cc-fa6f-4a97-bf3c-4952bf0e8145 => generated 109 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:00:36.516984 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:36.529918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8bd6c16e-1a1e-463f-ba9c-7b884e1b9a33 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.398405 May 23 13:00:36.530747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 67/124] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:36 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:36.724520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:36.725169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:36.734786 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:36.735249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:36.736052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:36.736479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:36.802881 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:36.804649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:36.811250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:36.811807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:36.812744 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:36.813175 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:36.914530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:36.923628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-684f21e3-266a-47c6-bfea-a92b56c2f3a4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.202716 May 23 13:00:36.932990 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 58/125] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:36 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:36.974500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:36.983426 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7b84c8e4-9b11-4a06-95b7-a3a58c2e8025 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.188007 May 23 13:00:36.984193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 68/126] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:36 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:37.547538 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:37.548155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:37.554087 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:37.554811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:37.555796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:37.556185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:37.726600 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:37.760022 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-6bb45c4e-062a-4df4-a7cb-c09be5719b5c req-f503a630-bc2c-4358-b57c-9fd81377e5ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf0c0b3-5059-405a-b847-07ed7c6edee9", "name": "network-vif-plugged", "status": "completed", "tag": "65ac147c-5bdb-47fa-adf1-5827d23a6e48"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:37.765191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b2f87f13-d5fd-4e63-95c7-cde74f4e3cf4 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.221477 May 23 13:00:37.776615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 59/127] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:37 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:37.791046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-6bb45c4e-062a-4df4-a7cb-c09be5719b5c req-f503a630-bc2c-4358-b57c-9fd81377e5ce service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:37.791477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-6bb45c4e-062a-4df4-a7cb-c09be5719b5c req-f503a630-bc2c-4358-b57c-9fd81377e5ce service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:37.809215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-6bb45c4e-062a-4df4-a7cb-c09be5719b5c req-f503a630-bc2c-4358-b57c-9fd81377e5ce service nova] Creating event network-vif-plugged:65ac147c-5bdb-47fa-adf1-5827d23a6e48 for instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 on hv-cinder-maste May 23 13:00:37.817415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-6bb45c4e-062a-4df4-a7cb-c09be5719b5c req-f503a630-bc2c-4358-b57c-9fd81377e5ce service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.179021 May 23 13:00:37.818052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 69/128] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:37.940629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:37.941266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:37.947368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:37.947836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:37.948592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:37.949015 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:38.002036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:38.002722 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:38.008870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:38.009402 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:38.010173 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:38.010637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:38.157971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:38.165822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1607897c-ea1e-4b2f-88c4-e3fcb2912315 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.229043 May 23 13:00:38.167211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 60/129] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:37 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:38.217106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:38.225173 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8c268d3d-1fcc-4380-b47a-6518f91bca0b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.227293 May 23 13:00:38.225749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 70/130] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:37 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:38.792660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:38.793268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:38.799582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:38.800040 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:38.800886 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:38.801324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:38.805432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:39.131213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:39.139114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ccaaa6da-dee7-4f08-ac05-a55c276b81f2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.350316 May 23 13:00:39.149643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 61/131] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:38 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:39.185388 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:39.186305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:39.195758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:39.196529 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:39.197419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:39.198512 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:39.243246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:39.243867 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:39.253388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:39.254519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:39.255377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:39.255898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:39.430996 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:39.439981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-24f4275c-c0e2-4f8e-add8-a048933dfd4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.260201 May 23 13:00:39.440917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 71/132] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:39 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:39.481245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:39.489021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-40b658df-6fc1-404d-94f7-421229180743 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.249702 May 23 13:00:39.489649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 62/133] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:39 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:40.160111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:40.160806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:40.167797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:40.168781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:40.169780 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:40.170481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:40.333767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:40.343639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-77d8d488-53c7-4963-9e57-5cb20ce083b6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.190127 May 23 13:00:40.345177 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 72/134] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:40 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:40.458781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:40.459794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:40.466882 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:40.467476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:40.468489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:40.469092 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:40.507497 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:40.508103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:40.517748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:40.518912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:40.519982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:40.520871 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:40.663662 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:40.672911 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9b188125-5c4d-421b-8f60-3c99c564e94f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.218558 May 23 13:00:40.682037 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 63/135] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:40 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:40.722138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:40.731851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f89d02a6-2b86-4a83-a76d-f0ed77a99717 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.228383 May 23 13:00:40.732653 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 73/136] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:40 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:41.365989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:41.367907 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:41.378719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.379963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.381570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.382680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.705698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:41.706944 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:41.713213 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.714492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.715673 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.716334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.727093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:41.746494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5a874710-8fcb-4451-b14f-e83a5532853f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.378243 May 23 13:00:41.746494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 64/137] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:41 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:41.750918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-49b78e88-5bb9-45b5-bd8f-a71670bc42fc req-79ce3033-c266-4c38-8a66-13bd3e3584e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "825e6db1-f2ed-47d4-88dc-07191da24f5f", "name": "network-vif-plugged", "status": "completed", "tag": "b2757ca5-c976-47ae-9a5d-46786a7e1473"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:41.768186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-49b78e88-5bb9-45b5-bd8f-a71670bc42fc req-79ce3033-c266-4c38-8a66-13bd3e3584e7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.768818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-49b78e88-5bb9-45b5-bd8f-a71670bc42fc req-79ce3033-c266-4c38-8a66-13bd3e3584e7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.791418 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-49b78e88-5bb9-45b5-bd8f-a71670bc42fc req-79ce3033-c266-4c38-8a66-13bd3e3584e7 service nova] Creating event network-vif-plugged:b2757ca5-c976-47ae-9a5d-46786a7e1473 for instance 825e6db1-f2ed-47d4-88dc-07191da24f5f on hv-cinder-maste May 23 13:00:41.805533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-49b78e88-5bb9-45b5-bd8f-a71670bc42fc req-79ce3033-c266-4c38-8a66-13bd3e3584e7 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064418 May 23 13:00:41.806183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 65/138] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:41.812588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:41.813936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:41.821451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.822206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.823746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:41.824675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:41.894029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:41.958155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:41.972472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a62f2bb5-d38f-4398-b500-b57242f55f6f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.272924 May 23 13:00:41.973736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 74/139] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:41 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:42.041705 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:42.061480 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0ebc6374-27c3-4940-bc74-3a95d530e579 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.253920 May 23 13:00:42.062177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 66/140] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:41 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:42.756205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:42.757350 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:42.765059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:42.765506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:42.766591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:42.767602 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:42.908255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:42.917439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9c7ba67d-cd0a-4730-a924-b769abee539f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.164817 May 23 13:00:42.918040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 75/141] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:42 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:42.989799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:42.990494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:42.999724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:43.000427 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:43.001726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:43.002338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:43.080090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:43.080733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:43.086888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:43.087561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:43.088487 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:43.089571 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:43.198444 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:43.206628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6840edb0-64ee-4fd9-aaee-b536a891cade tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.220626 May 23 13:00:43.207365 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 67/142] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:42 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:43.254821 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:43.263032 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0176833d-b69d-46cc-a07b-61fc19753549 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.186800 May 23 13:00:43.263585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 76/143] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:43 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:43.935045 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:43.935780 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:43.942162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:43.946046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:43.946046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:43.946046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:44.072245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:44.080246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-70e6caae-7e76-4f6b-b3e8-8ad86cce0f3b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.149633 May 23 13:00:44.090461 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 68/144] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:43 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:44.224368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:44.230647 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:44.232677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:44.233135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:44.234628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:44.235561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:44.282359 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:44.283160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:44.297603 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:44.298181 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:44.299291 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:44.304399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:44.610926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:44.618840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7e884495-2543-4a91-9e91-0a5c8a02dc5a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.398293 May 23 13:00:44.619506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 77/145] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:44 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:44.638398 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:44.645716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-63abf235-77a5-4958-a4da-6ea97a2d1dbc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.369834 May 23 13:00:44.646314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 69/146] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:44 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:45.097667 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:45.098297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:45.104301 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:45.104741 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:45.105484 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:45.105941 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:45.238161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:45.248368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8281206f-65ff-46c1-9d78-2589bbb98ffc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.154458 May 23 13:00:45.249504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 78/147] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:45 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:45.636827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:45.638303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:45.644429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:45.644881 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:45.645647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:45.646229 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:45.662529 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:45.663302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:45.670161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:45.670993 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:45.671737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:45.672222 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:45.831994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:45.840404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-506da364-be5f-4f83-ad06-e7dd1a4d1fd9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.207956 May 23 13:00:45.841558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 70/148] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:45 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:45.861656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:45.869093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3b25e0aa-d8a2-4121-8894-b90b0e22dd4e tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.210496 May 23 13:00:45.870746 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 79/149] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:45 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:46.266932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:46.267534 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:46.273619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:46.274271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:46.275034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:46.275465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:46.473310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:46.478677 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ad5c78d4-013f-4434-ad40-7361ebef6a44 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.215532 May 23 13:00:46.488770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 71/150] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:46 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:46.859893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:46.860519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:46.874487 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:46.878075 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:46.880417 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:46.883239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:46.886860 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:46.887726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:46.897129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:46.897579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:46.898385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:46.898818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:47.328456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:47.331491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:47.339998 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c3fcd67d-d4b3-482c-875d-25bc174460b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.457843 May 23 13:00:47.349251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 72/151] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:46 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:47.361682 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-58bdef3c-a36f-4fd2-be75-ad10880849e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.506517 May 23 13:00:47.362556 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 80/152] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:46 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:47.498384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:47.499418 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:47.507675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:47.508360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:47.509606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:47.521807 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:47.670266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:47.682228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-42e410d9-0bd6-4f7e-b334-9b445e4af97c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.188683 May 23 13:00:47.684044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 73/153] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:47 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:48.357433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:48.358073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:48.364023 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:48.364498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:48.365385 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:48.365815 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:48.378899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:48.379521 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:48.386247 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:48.387472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:48.387768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:48.388207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:48.589240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:48.597496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-312f0ffa-f36f-485a-bf4a-ea80726eb4fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.244729 May 23 13:00:48.598200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 81/154] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:48 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:48.607023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:48.615384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e1108573-5af7-4ad4-93f8-e10e4ae31443 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.240206 May 23 13:00:48.616150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 74/155] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:48 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:48.701037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:48.701783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:48.707874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:48.708392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:48.709104 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:48.709558 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:48.851818 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:48.860212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fe66c1bf-52c4-4a4d-a060-cdf7acdf0aca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.163444 May 23 13:00:48.860788 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 82/156] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:48 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:49.615106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:49.615676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:49.622015 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:49.622468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:49.623263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:49.623760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:49.632459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:49.639307 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:49.640665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:49.641114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:49.641893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:49.642329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:49.826523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:49.835032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2d86036d-ee1b-4c98-be05-f3f8ee0ca046 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.223625 May 23 13:00:49.836294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 75/157] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:49 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:49.839915 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:49.847848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-aa182abe-2d8d-45a9-839c-d4b9f61e69c1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.219228 May 23 13:00:49.848400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 83/158] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:49 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:49.878093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:49.878858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:49.884670 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:49.885229 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:49.886377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:49.886876 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:50.021084 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:50.030781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-000dfe10-dd7b-4b4b-bb70-59b356cfe09f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.156569 May 23 13:00:50.031675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 76/159] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:49 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:50.856002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:50.856613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:50.863014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:50.863467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:50.864212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:50.864613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:50.864702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:50.864804 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:50.869933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=71 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:50.871621 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:50.872063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:50.872832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:50.873273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:50.884561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:51.070006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:51.071819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:51.078115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6f4db92f-37a7-4a81-a968-6b35ec8e19eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.217565 May 23 13:00:51.078682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 77/160] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:50 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:51.083797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:51.084393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:51.089176 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-73a9835b-b7f7-491c-a907-2bb9d5fdbb5b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.238058 May 23 13:00:51.094445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 84/161] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:50 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:51.094627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:51.094627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:51.094627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:51.094627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:51.115326 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:51.226268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:51.234100 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e29c9809-b2bd-4e65-8730-d6c800dd37b7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.154100 May 23 13:00:51.234666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 78/162] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:51 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:52.094769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:52.095426 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:52.101369 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:52.101980 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:52.102842 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:52.103279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:52.106505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:52.107116 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:52.113731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:52.114210 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:52.114974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:52.115412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:52.297233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:52.305304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7dd60f85-5787-4401-be9e-6a529f69ca8f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.214212 May 23 13:00:52.305894 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 85/163] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:52 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:52.311598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:52.312448 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:52.312854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:52.319253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:52.319698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:52.320454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:52.320938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:52.323541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6e9df0a7-3c75-4c25-b5a3-c35e0abec036 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.220959 May 23 13:00:52.323788 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 79/164] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:52 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:52.454343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:52.461708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d4eace38-934d-4dd2-970a-d5ad338fd4e6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.154877 May 23 13:00:52.462315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 86/165] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:52 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:53.323267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:53.323713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:53.330709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.331162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.332067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.332508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.341782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:53.343374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:53.352053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.353167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.355119 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.355950 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.565171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:53.585491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1d99b235-6a9e-4a70-be38-86ebbdf9b121 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.243179 May 23 13:00:53.585491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 87/166] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:53 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:53.585632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-4cdcbb88-d807-47c0-9b15-44e317e655dc req-af4251a4-2aef-490f-a163-8c06124a6f4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e01afbdf-8ced-4f91-b14c-b87b2fcf1506", "name": "network-vif-plugged", "status": "completed", "tag": "97ab468b-d2e1-4b56-98d6-23dfeb309dc3"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:53.596044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4cdcbb88-d807-47c0-9b15-44e317e655dc req-af4251a4-2aef-490f-a163-8c06124a6f4d service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.596981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4cdcbb88-d807-47c0-9b15-44e317e655dc req-af4251a4-2aef-490f-a163-8c06124a6f4d service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.615212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-4cdcbb88-d807-47c0-9b15-44e317e655dc req-af4251a4-2aef-490f-a163-8c06124a6f4d service nova] Creating event network-vif-plugged:97ab468b-d2e1-4b56-98d6-23dfeb309dc3 for instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 on hv-cinder-maste May 23 13:00:53.622924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-4cdcbb88-d807-47c0-9b15-44e317e655dc req-af4251a4-2aef-490f-a163-8c06124a6f4d service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040972 May 23 13:00:53.623735 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 88/167] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:00:53.630121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:53.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:53.641076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.641768 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.642929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:53.643526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:53.647745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:53.656238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-792495de-63d7-4ce6-8497-352a5d9f27ab tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1321 microversion: 2.1 time: 0.337263 May 23 13:00:53.657306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 80/168] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:53 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1321 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:53.789170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:53.796634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6d7847c4-e517-47e7-9ba7-6b7545bb4bfe tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.171624 May 23 13:00:53.797202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 89/169] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:53 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:54.600596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:54.601198 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:54.608089 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:54.608734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:54.610300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:54.610999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:54.674105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:54.674843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:54.681540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:54.682263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:54.683754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:54.684302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:54.802689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:54.813519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c982517d-6ccc-472a-aaee-697adc727cdc tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1314 microversion: 2.1 time: 0.216399 May 23 13:00:54.814403 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 81/170] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:54 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:54.825688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:54.825688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:54.834058 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:54.834565 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:54.835337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:54.835770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.065453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:55.078203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0c5eafb4-e438-4e38-a260-5aa6aa2e393c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1515 microversion: 2.1 time: 0.408069 May 23 13:00:55.082190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 90/171] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:54 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1515 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:55.100755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:55.102163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:55.145600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.148263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.149303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.152195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.227766 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:55.236056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-579aa976-3a0d-44b6-a56e-b88fcde32b37 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.419957 May 23 13:00:55.236628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 82/172] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:54 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:55.250891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lazy-loading 'flavor' on Instance uuid fbf0c0b3-5059-405a-b847-07ed7c6edee9 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:00:55.643772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-63df257a-672e-4dd9-a936-bb4f776ef3f3 req-70f6c6fb-99b0-45d5-8993-337f52964377 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d4562fc-a49b-4d0d-b486-e8d2fb59db7c", "name": "network-vif-plugged", "status": "completed", "tag": "381aa96b-0ff9-4824-b7af-8578a2b139b2"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:55.656039 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-63df257a-672e-4dd9-a936-bb4f776ef3f3 req-70f6c6fb-99b0-45d5-8993-337f52964377 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.656527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-63df257a-672e-4dd9-a936-bb4f776ef3f3 req-70f6c6fb-99b0-45d5-8993-337f52964377 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.668413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-63df257a-672e-4dd9-a936-bb4f776ef3f3 req-70f6c6fb-99b0-45d5-8993-337f52964377 service nova] Creating event network-vif-plugged:381aa96b-0ff9-4824-b7af-8578a2b139b2 for instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c on hv-cinder-maste May 23 13:00:55.673439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-63df257a-672e-4dd9-a936-bb4f776ef3f3 req-70f6c6fb-99b0-45d5-8993-337f52964377 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033459 May 23 13:00:55.674239 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 83/173] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:00:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:00:55.768823 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c5eace48-8b4d-4891-bcff-4cf08e801adc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9/action" status: 200 len: 32 microversion: 2.1 time: 0.675378 May 23 13:00:55.769496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 91/174] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:00:55 2022] POST /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9/action => generated 32 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 23 13:00:55.783677 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:55.784365 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:55.790827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.791271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.801528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.803825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.842200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:55.846010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:55.862263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.862720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:55.865611 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:55.866293 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:56.160792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:56.169360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fff61bae-b4cb-4824-b1e3-bb81b7937f36 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.389820 May 23 13:00:56.170274 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 84/175] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:55 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:56.192072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:56.200267 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-81328876-6843-4aa6-b0b1-5ff9b5ffa836 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1501 microversion: 2.1 time: 0.369313 May 23 13:00:56.200831 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 92/176] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:55 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1501 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:56.252282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:56.252889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:56.258740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:56.259180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:56.259933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:56.260364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:56.595314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:56.604030 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9a7edaee-ed15-4270-a847-399af431876f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.355587 May 23 13:00:56.610347 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 85/177] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:56 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:57.186108 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:57.186747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:57.188733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=76 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:00:57.193647 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:57.194133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:57.194907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:57.195336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:57.331473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:57.340089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0907dd18-331b-4514-ba2d-33c9454cc60f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.157790 May 23 13:00:57.340648 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 93/178] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:57 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:57.626689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:57.627863 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:57.634224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:57.634664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:57.635514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:57.636752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:57.773681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:57.782501 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0fe0f6bf-20df-4fa8-9a6e-17197b232b75 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.159846 May 23 13:00:57.783079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 86/179] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:57 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:00:58.357486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:58.361901 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:58.365092 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:58.365690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:58.366351 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:58.369551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:58.520143 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:58.531929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f19255f6-4774-4113-9209-5a9d743ca76a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.178793 May 23 13:00:58.532496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 94/180] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:58 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:58.799251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:58.799786 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:58.806436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:58.806958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:58.808335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:58.808829 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:59.000656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:59.009489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4323fdde-9673-466e-83bf-cc4422e0149b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1366 microversion: 2.1 time: 0.214106 May 23 13:00:59.010350 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 87/181] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:58 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1366 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:00:59.214309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b55ba38c-c6dc-4591-9759-701ea3d76463"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:00:59.216129 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:59.224757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:59.225197 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:59.226164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:59.226589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:59.548967 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:00:59.549570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:00:59.555653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:59.556107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:59.556884 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:00:59.557312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:00:59.719784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:00:59.739869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-49b1f002-668e-45a6-b3e6-93fdb9478d05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.194390 May 23 13:00:59.750003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 88/182] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:00:59 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:00.034958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:00.037732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:00.047193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:00.047632 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:00.049360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:00.049983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:00.196524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:00.204972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e2c7ba01-e1c2-48b6-85e5-ecec29a86dea tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1556 microversion: 2.1 time: 0.179070 May 23 13:01:00.205429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 89/183] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:00 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1556 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:00.222272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:00.222872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:00.231723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:00.231723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:00.231723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:00.231723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:00.349828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b5df751c-d9d0-464d-affa-aea710519679 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.131384 May 23 13:01:00.356356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 90/184] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:01:00 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-interface => generated 285 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:01:00.760250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:00.760978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:00.777111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:00.787404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:00.787404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:00.787404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:01.315287 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3ad91a28-5966-49ce-8d74-55baa98c4446 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "POST /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.105529 May 23 13:01:01.316335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 95/185] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:00:59 2022] POST /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f/os-volume_attachments => generated 194 bytes in 2108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:01.340770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:01.349546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1db3a6b5-025a-4714-abbe-375450a5d5af tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.594503 May 23 13:01:01.350480 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 91/186] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:00 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:02.113980 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-8ae2e638-4a89-4b6f-b101-6b6237437b58 req-c1273eef-485c-4e52-9939-0d2a2b8808be service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e01afbdf-8ced-4f91-b14c-b87b2fcf1506", "tag": "97ab468b-d2e1-4b56-98d6-23dfeb309dc3"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:02.123209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8ae2e638-4a89-4b6f-b101-6b6237437b58 req-c1273eef-485c-4e52-9939-0d2a2b8808be service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:02.123960 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8ae2e638-4a89-4b6f-b101-6b6237437b58 req-c1273eef-485c-4e52-9939-0d2a2b8808be service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:02.136845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-8ae2e638-4a89-4b6f-b101-6b6237437b58 req-c1273eef-485c-4e52-9939-0d2a2b8808be service nova] Creating event network-changed:97ab468b-d2e1-4b56-98d6-23dfeb309dc3 for instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 on hv-cinder-maste May 23 13:01:02.142810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-8ae2e638-4a89-4b6f-b101-6b6237437b58 req-c1273eef-485c-4e52-9939-0d2a2b8808be service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033595 May 23 13:01:02.143645 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 96/187] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:01:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:02.369993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:02.371670 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:02.379477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:02.379931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:02.381211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:02.381656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:02.745539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:02.747144 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2d3c8ce9-c7e2-4cfb-a1b6-21ec1aa9433a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.381712 May 23 13:01:02.748138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 92/188] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:02 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:03.764925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:03.765625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:03.772278 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:03.772730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:03.773474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:03.773942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:03.923809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:03.931150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b39df6ea-9737-4481-a682-cda62b513e86 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.170827 May 23 13:01:03.931711 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 97/189] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:03 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:04.950239 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:04.950965 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:04.963094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:04.963453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:04.964905 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:04.965513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:05.110167 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:05.118302 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6f929263-d837-4586-9fb5-bd83d66a208d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.173555 May 23 13:01:05.119026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 93/190] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:04 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:06.136852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:06.138351 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:06.146033 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:06.146730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:06.147492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:06.148415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:06.295547 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:06.303285 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-969a245e-fe5c-4dbe-a0e2-8e42ed0b1eeb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.170682 May 23 13:01:06.303845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 98/191] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:06 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:07.320779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:07.322236 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:07.329119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:07.329568 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:07.333854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:07.333854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:07.465490 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:07.472933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-813f0eb8-cec8-4f00-bb76-4d07f2cd2301 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.156172 May 23 13:01:07.473920 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 94/192] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:07 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:08.490169 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:08.490776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:08.498160 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:08.498616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:08.499754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:08.500205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:08.504499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:08.669211 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:08.686476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a9679f18-e225-4cdd-86fb-b96d2c6d5234 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.199450 May 23 13:01:08.687956 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 99/193] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:08 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:09.704701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:09.705313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:09.712369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:09.712819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:09.713580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:09.714763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:09.859479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:09.867455 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e3eca917-a957-40d2-af6d-1837c682a1ed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.166778 May 23 13:01:09.868514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 95/194] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:09 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:10.885156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:10.887166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:10.894536 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:10.894974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:10.895724 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:10.896148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:11.086209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:11.094527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c86d4218-dd65-4ff8-8dfc-9196145a31d8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.213531 May 23 13:01:11.095540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 100/195] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:10 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:12.114729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:12.116030 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:12.127361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:12.127827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:12.128601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:12.129032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:12.133393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:12.430529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:12.452216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6c8609f1-c022-43a9-be56-c35dcc4a8eb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.341150 May 23 13:01:12.458247 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 96/196] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:12 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:13.470655 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:13.471895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:13.481663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:13.482148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:13.489156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:13.489156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:13.670726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:13.686299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-76174929-b31f-44d4-a439-40144dca83e9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.220110 May 23 13:01:13.697165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 101/197] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:13 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:13.856385 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:13.858013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:13.866591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:13.890314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:13.892963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:13.892963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:13.974630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:01:14.528985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cf6e5db3-b8a0-437d-bfa0-5fb91d2edaac tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "DELETE /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 204 len: 0 microversion: 2.1 time: 0.677748 May 23 13:01:14.530716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 97/198] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:01:13 2022] DELETE /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 0 bytes in 681 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:01:14.539521 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:14.540126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:14.556449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:14.556449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:14.556449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:14.556449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:14.721787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:14.723120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:14.749554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:14.752836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:14.755111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:14.756432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:14.785693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:14.798211 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f391d9cb-e434-40e5-9350-ea1e0184b071 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1553 microversion: 2.1 time: 0.261636 May 23 13:01:14.798211 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 102/199] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:14 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1553 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:14.960183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:14.968495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-40c55da0-6505-4a14-9795-783a42a96d04 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.253398 May 23 13:01:14.969447 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 98/200] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:14 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:15.823740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:15.825241 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:15.832385 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:15.833171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:15.834105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:15.834850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:15.987404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:15.988023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:15.999652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:16.000372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:16.001410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:16.002345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:16.460916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:16.472500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3dfe357c-9751-4458-9546-e8005490062b tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1553 microversion: 2.1 time: 0.650776 May 23 13:01:16.474520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 103/201] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:15 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1553 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:16.526337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:16.535607 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2b5c78f0-d02f-4ebf-95e4-f9e9b6db9b51 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.552402 May 23 13:01:16.546645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 99/202] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:15 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:17.491733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:17.492347 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:17.506481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:17.507111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:17.507900 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:17.508582 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:17.558457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:17.558910 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:17.570011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:17.572292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:17.573294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:17.575410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:17.972258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:17.983872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ef184101-681a-42cf-b99e-5f95152c3b49 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.431220 May 23 13:01:17.985643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 100/203] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:17 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:17.995971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:18.015879 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dbe610a7-53cc-4a62-8b24-a54a2719a5fd tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1553 microversion: 2.1 time: 0.527821 May 23 13:01:18.020923 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 104/204] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:17 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1553 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:18.557555 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-fe3532f5-ed3e-45b8-8504-cb068c7e65e1 req-b69ffefc-6e45-4263-be0a-f65f2642c0ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "825e6db1-f2ed-47d4-88dc-07191da24f5f", "name": "network-vif-deleted", "tag": "b2757ca5-c976-47ae-9a5d-46786a7e1473"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:18.594659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-fe3532f5-ed3e-45b8-8504-cb068c7e65e1 req-b69ffefc-6e45-4263-be0a-f65f2642c0ff service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:18.596044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-fe3532f5-ed3e-45b8-8504-cb068c7e65e1 req-b69ffefc-6e45-4263-be0a-f65f2642c0ff service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:18.613353 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-fe3532f5-ed3e-45b8-8504-cb068c7e65e1 req-b69ffefc-6e45-4263-be0a-f65f2642c0ff service nova] Creating event network-vif-deleted:b2757ca5-c976-47ae-9a5d-46786a7e1473 for instance 825e6db1-f2ed-47d4-88dc-07191da24f5f on hv-cinder-maste May 23 13:01:18.619227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-fe3532f5-ed3e-45b8-8504-cb068c7e65e1 req-b69ffefc-6e45-4263-be0a-f65f2642c0ff service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069516 May 23 13:01:18.620272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 101/205] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:01:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:19.002878 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:19.004518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:19.018869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:19.019641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:19.020418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:19.021028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:19.038267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:19.038960 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:19.045407 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:19.045857 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:19.046821 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:19.047520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:19.150330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:19.162934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-764f7fa9-98f3-4880-b53f-b89e25b70414 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1345 microversion: 2.1 time: 0.128738 May 23 13:01:19.163517 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 102/206] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:19 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1345 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:19.348473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:19.376259 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-849df5f6-d53f-4320-9d48-d6b59106cfce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.377852 May 23 13:01:19.383500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 105/207] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:18 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:20.180121 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:20.180733 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:20.195656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:20.200806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:20.202682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:20.203514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:20.338035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:20.351665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cfc85a0b-eda5-4b12-9911-ab83f266b0b9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1345 microversion: 2.1 time: 0.175233 May 23 13:01:20.353760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 103/208] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:20 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1345 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:20.395053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:20.395668 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:20.408515 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:20.408967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:20.410079 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:20.410515 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:20.662204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:20.671949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6fea03bb-706d-4e3e-a349-921f662b32eb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.280789 May 23 13:01:20.673129 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 106/209] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:20 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:20.987234 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1214877407", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "networks": [{"uuid": "58c39c65-d0c2-47c3-84c7-3d9e660a4739"}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:20.996373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=59,nova_api:UPDATE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:21.063776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] validate_networks() for [('58c39c65-d0c2-47c3-84c7-3d9e660a4739', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:01:21.372907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:21.373691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:21.385845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.387001 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:21.387789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.391184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:21.468507 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.469163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:21.474591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lazy-loading 'fault' on Instance uuid 825e6db1-f2ed-47d4-88dc-07191da24f5f {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:01:21.745702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b80726b3f14828b8815e30fc7e13c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9e3c84deee9a435a9c1e72e8408c18b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:21.753842 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2d366703-1df7-4400-b8e8-5653f88f81c7 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 200 len: 1304 microversion: 2.1 time: 0.386882 May 23 13:01:21.759440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 107/210] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:21 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 1304 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:21.784464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:21.786475 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:21.809597 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.818579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:21.819504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.819959 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:21.961981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:01:21.963395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:01:21.964696 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:01:21.966630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:01:21.967307 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:01:21.984860 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.985306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:21.986200 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:21.986629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:22.014791 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Getting quotas for project 3b3719ab772449d79dc00dc96487dc36. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:01:22.019335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Getting quotas for user 7f5e411f4d8e4f13b662cf2846720cd3 and project 3b3719ab772449d79dc00dc96487dc36. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:01:22.033712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:01:22.036448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:01:22.061374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:22.069926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4c64b861-aedb-4086-8787-458d020f595c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.309153 May 23 13:01:22.070591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 108/211] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:21 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:22.216584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d4b47cf7-69ca-4410-9535-e4107fd402ef tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.240233 May 23 13:01:22.217865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 104/212] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:01:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 1242 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:01:22.230144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a1c3a3af-e952-4710-8142-b2d6abf10a0c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:22.230998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a1c3a3af-e952-4710-8142-b2d6abf10a0c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:22.304634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a1c3a3af-e952-4710-8142-b2d6abf10a0c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:22.321553 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a1c3a3af-e952-4710-8142-b2d6abf10a0c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1214 microversion: 2.1 time: 0.096980 May 23 13:01:22.322602 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 109/213] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:22 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1214 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:22.771833 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:22.772772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:22.782099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:22.785298 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:22.786728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:22.788394 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:22.813611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] HTTP exception thrown: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Request method failure captured: May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f HTTP/1.1 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VMU1MR0Ql8sfry8dk7LAg8X12UQOzNsefFpyiLfdOBr5nvtA5cag42RoqxmNmiXsQfEr-ZhJr1BUE0epJFEKHjdmDODR2uqPZGrDPn-bgRfLjRWg0O9brw870yPWQLg6j8rNU_d_gfbkoj66aibWOFk4bd4vWWHbULj1w3W4J9i14 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:01:22.817724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: a9b80726b3f14828b8815e30fc7e13c5 May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:22.819263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '825e6db1-f2ed-47d4-88dc-07191da24f5f'} May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:01:22.820095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:22.821617 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:01:22.824237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Returning 404 to user: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:01:22.824237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bf40d4ff-7059-4c6c-b766-546b47ded4d9 tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 404 len: 111 microversion: 2.1 time: 0.051589 May 23 13:01:22.824237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 105/214] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:22 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:01:22.999150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:23.001205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:23.010218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:23.011406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:23.012809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:23.013739 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:23.029120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] HTTP exception thrown: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Request method failure captured: May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f/os-volume_attachments HTTP/1.1 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VMU1MR0Ql8sfry8dk7LAg8X12UQOzNsefFpyiLfdOBr5nvtA5cag42RoqxmNmiXsQfEr-ZhJr1BUE0epJFEKHjdmDODR2uqPZGrDPn-bgRfLjRWg0O9brw870yPWQLg6j8rNU_d_gfbkoj66aibWOFk4bd4vWWHbULj1w3W4J9i14 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:01:23.034612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: a9b80726b3f14828b8815e30fc7e13c5 May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:23.035910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'server_id': '825e6db1-f2ed-47d4-88dc-07191da24f5f'} May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:01:23.036578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:23.037654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:23.040399 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Returning 404 to user: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:01:23.041486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-17bb1146-0924-4aac-9d2f-59e267ab438a tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.048404 May 23 13:01:23.046326 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 110/215] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:01:22 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f/os-volume_attachments => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:01:23.091199 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:23.092581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:23.102550 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:23.103009 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:23.103791 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:23.104227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:23.339899 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-36d203a6-d444-438b-bf65-94b42a7d4ea1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:23.340980 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-36d203a6-d444-438b-bf65-94b42a7d4ea1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:23.536162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-36d203a6-d444-438b-bf65-94b42a7d4ea1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:23.544357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-36d203a6-d444-438b-bf65-94b42a7d4ea1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1214 microversion: 2.1 time: 0.208332 May 23 13:01:23.547105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 111/216] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:23 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1214 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:23.583000 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:23.599456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4a582cf5-1541-48f2-9439-b5d27e6d0d2c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.515535 May 23 13:01:23.604473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 106/217] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:23 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:24.564562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:24.565193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:24.578194 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.578972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.579983 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.583133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.622512 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:24.623279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:24.652987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.653481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.654665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.655119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.738773 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:24.747510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-79c4fab5-e101-4b46-8cf2-ea52c436d34c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1266 microversion: 2.1 time: 0.187049 May 23 13:01:24.750180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 112/218] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:24 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1266 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:24.755129 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:24.755826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:24.763077 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.764385 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.765334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.765983 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.800997 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] HTTP exception thrown: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Request method failure captured: May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: DELETE /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f HTTP/1.1 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VMU1MR0Ql8sfry8dk7LAg8X12UQOzNsefFpyiLfdOBr5nvtA5cag42RoqxmNmiXsQfEr-ZhJr1BUE0epJFEKHjdmDODR2uqPZGrDPn-bgRfLjRWg0O9brw870yPWQLg6j8rNU_d_gfbkoj66aibWOFk4bd4vWWHbULj1w3W4J9i14 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:01:24.803470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: a9b80726b3f14828b8815e30fc7e13c5 May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.804666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '825e6db1-f2ed-47d4-88dc-07191da24f5f'} May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:01:24.805330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.806621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.809564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Returning 404 to user: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:01:24.810643 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5f4d1f5c-e868-4fa7-9a4d-f248dbdc9bda tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "DELETE /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 404 len: 111 microversion: 2.1 time: 0.059372 May 23 13:01:24.811386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 113/219] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:01:24 2022] DELETE /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:01:24.819070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:24.819779 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] [instance: 825e6db1-f2ed-47d4-88dc-07191da24f5f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:24.826677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.827127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.827936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:24.828987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:24.847487 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] HTTP exception thrown: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Request method failure captured: May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f HTTP/1.1 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VMU1MR0Ql8sfry8dk7LAg8X12UQOzNsefFpyiLfdOBr5nvtA5cag42RoqxmNmiXsQfEr-ZhJr1BUE0epJFEKHjdmDODR2uqPZGrDPn-bgRfLjRWg0O9brw870yPWQLg6j8rNU_d_gfbkoj66aibWOFk4bd4vWWHbULj1w3W4J9i14 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:01:24.849188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/9e3c84deee9a435a9c1e72e8408c18b4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 9e3c84deee9a435a9c1e72e8408c18b4 May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-DeleteServersTestJSON-1673527158 May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: a9b80726b3f14828b8815e30fc7e13c5 May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-DeleteServersTestJSON-1673527158-project May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.850710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '825e6db1-f2ed-47d4-88dc-07191da24f5f'} May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:01:24.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. May 23 13:01:24.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:01:24.855649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] Returning 404 to user: Instance 825e6db1-f2ed-47d4-88dc-07191da24f5f could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:01:24.856858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5520cdb7-6f7b-406a-a331-e7b2979389ae tempest-DeleteServersTestJSON-1673527158 tempest-DeleteServersTestJSON-1673527158-project] 10.222.0.30 "GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f" status: 404 len: 111 microversion: 2.1 time: 0.041590 May 23 13:01:24.862518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 114/220] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:24 2022] GET /compute/v2.1/servers/825e6db1-f2ed-47d4-88dc-07191da24f5f => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 13:01:25.039985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:25.048877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bb792bd7-30f8-424b-8e91-f7b44c9cc567 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.431983 May 23 13:01:25.052803 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 107/221] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:24 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:25.769286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:25.770584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:25.779559 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:25.780246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:25.781698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:25.782312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:25.877789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:25.888795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-40f6f38c-8d2b-4b54-a5a8-301c85e90ca0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1272 microversion: 2.1 time: 0.123966 May 23 13:01:25.889385 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 115/222] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:25 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1272 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:26.069137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:26.069901 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:26.080296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:26.085657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:26.093592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:26.095427 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:26.578892 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:26.600125 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-97548d52-7399-4f9c-82d0-f3ccb4c14eaf tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.535370 May 23 13:01:26.600726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 108/223] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:26 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:26.911253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:26.913624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:26.926588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:26.927489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:26.928772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:26.933228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:27.152340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:27.162436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-54e5291c-dddb-481e-a2a8-a83133bbfbdc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.259090 May 23 13:01:27.163612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 109/224] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:26 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:27.329148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d5016b2a-92eb-4ecf-8a82-804609ccb0f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Action: 'create', calling method: .version_select at 0x7f9b180a9dc0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1848200419", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNZBsFBFQjddrUBKLvg0AQGykI0l7r8guSwsWN7KzHSRyss/u45Nh6mfg/HM3U1v2q6SFX7GCqmOQ4+EKMLNFDoUqywJNCVg3qptv9M/TRf4NgiCh2YoqNJU2aHmeO5OCw=="}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:27.335455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=52 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:27.346511 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-d5016b2a-92eb-4ecf-8a82-804609ccb0f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:01:27.351676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-d5016b2a-92eb-4ecf-8a82-804609ccb0f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for user c6a6096760734ac49ce2f26c6065b701 and project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:01:27.484720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d5016b2a-92eb-4ecf-8a82-804609ccb0f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.606905 May 23 13:01:27.485781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 116/225] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:01:26 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:27.629708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:27.629708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:27.636212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:27.636660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:27.637454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:27.637944 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:27.869302 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:27.878012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-12d1c565-61c4-4f3b-abe9-4598ea8316f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.258641 May 23 13:01:27.878594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 110/226] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:27 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:28.181028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:28.182030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:28.192881 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:28.193568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:28.194686 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:28.195293 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:28.553276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:28.562383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5a9488b5-faf5-4142-bcfc-584385b76395 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.385240 May 23 13:01:28.576696 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 117/227] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:28 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:28.896421 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:28.897025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:28.906548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:28.907207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:28.908128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:28.908883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:29.059488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:29.068074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b26fe705-f042-44f6-80f2-f5f32392e11d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.175552 May 23 13:01:29.078367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 111/228] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:28 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:29.587255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:29.588515 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:29.598220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:29.598989 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:29.599972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:29.601325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:29.986870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:30.019766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-029a30aa-cf24-4b1e-8820-be73cf12a2f6 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.438699 May 23 13:01:30.022113 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 118/229] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:29 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:30.085208 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:30.086455 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:30.094000 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:30.094369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:30.095153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:30.095627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:30.427614 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:30.448014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-992bf4fa-d650-4224-81d9-eff5081ddc69 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.367122 May 23 13:01:30.449202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 112/230] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:30 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:31.040690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:31.041295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:31.048757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:31.050502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:31.050502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:31.050502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:31.352294 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:31.361421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3601fcef-72c6-4d65-8bdd-e840995d1777 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.324610 May 23 13:01:31.373231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 119/231] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:31 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:31.465664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:31.466492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:31.477681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:31.478175 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:31.478961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:31.479434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:31.488030 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:31.706823 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:31.726182 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-fd819b23-9635-4c34-a005-d4804c76e87c req-d44cab37-fab2-40d2-9282-bd04bca0e29b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8", "tag": "01e9d1a4-308e-43f5-9c0e-42aa73aab730"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:31.747318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6ac48f0a-b512-4413-bf8c-90f658d4fbe0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.272149 May 23 13:01:31.749713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 113/232] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:31 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:31.752185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-fd819b23-9635-4c34-a005-d4804c76e87c req-d44cab37-fab2-40d2-9282-bd04bca0e29b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:31.756026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-fd819b23-9635-4c34-a005-d4804c76e87c req-d44cab37-fab2-40d2-9282-bd04bca0e29b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:31.786341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-fd819b23-9635-4c34-a005-d4804c76e87c req-d44cab37-fab2-40d2-9282-bd04bca0e29b service nova] Creating event network-changed:01e9d1a4-308e-43f5-9c0e-42aa73aab730 for instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 on hv-cinder-maste May 23 13:01:31.795719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-fd819b23-9635-4c34-a005-d4804c76e87c req-d44cab37-fab2-40d2-9282-bd04bca0e29b service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074056 May 23 13:01:31.795719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 120/233] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:01:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:32.383799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:32.384836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:32.388879 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:32.390064 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:32.394338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:32.394787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:32.395728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:32.396428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:32.717479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:32.724815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c9986e61-ab92-45c6-aa04-3d7f7cfa798e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.348420 May 23 13:01:32.733940 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 114/234] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:32 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:32.768096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:32.772063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:32.781639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:32.783617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:32.787053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:32.789089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:32.794239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:33.023016 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:33.036638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-006e5237-5365-42d8-b802-28871ef900dc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.275596 May 23 13:01:33.037219 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 121/235] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:32 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:33.742289 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:33.742835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:33.761682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:33.762157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:33.762941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:33.763373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:34.014499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:34.031472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-09c1ae66-3d07-4786-8d8d-88e9d7931291 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.292869 May 23 13:01:34.033999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 115/236] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:33 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:34.054811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:34.055411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:34.062610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:34.063082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:34.063861 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:34.064328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:34.310868 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:34.322823 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-239efdb4-afe7-4c70-a384-521f5b20a526 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.271950 May 23 13:01:34.345991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 122/237] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:34 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:35.049192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:35.049790 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:35.056543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:35.057190 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:35.058414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:35.059016 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:35.344279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:35.345866 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:35.353998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:35.355070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:35.356415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:35.357756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:35.580493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:35.588897 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a121da2d-c38c-41a0-a0a8-b02f44c09731 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.543518 May 23 13:01:35.591683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 116/238] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:35 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:35.741538 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:35.750477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5bfe4f27-c0b5-4ba7-a691-7e7b95d0afec tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.411319 May 23 13:01:35.751400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 123/239] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:35 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:36.608364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:36.609134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:36.617678 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:36.618931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:36.619798 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:36.620912 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:36.769756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:36.780311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:36.786935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:36.793625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:36.794749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:36.795336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:37.161986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:37.173819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-615c3f8b-0ec3-48f6-beda-0b9fb8bcc95c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.570010 May 23 13:01:37.177264 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 117/240] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:36 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:37.207393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:37.217197 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6ff95e1b-68b2-4eed-a098-862b5a38f6c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.451201 May 23 13:01:37.218274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 124/241] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:36 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:38.196280 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:38.200667 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:38.223198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=8 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:38.230647 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:38.232832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:38.233852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:38.234317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:38.453137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1d2f2644-a3f7-4621-96ba-b6d73c614578 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'create', calling method: .version_select at 0x7f9b1809da60>, body: {"keypair": {"name": "tempest-keypair-596350624", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGY2KrdQ+ngKcEVa+lmlq/FfnEaCsYmOPypjIVEP+189EHU+lzVjAH2vysFR3/mTybLbxM0BAhxTbTCPb101zEHPhETDMTla3XWuhs1vHvY3dl7HF6WnG9vs1aaToNjHRQ=="}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:38.454758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:38.492895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a461ca48-dae2-4f05-9675-f0240ba942ee tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.303835 May 23 13:01:38.496156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 125/242] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:38 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:38.498055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-1d2f2644-a3f7-4621-96ba-b6d73c614578 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for project 5b4d791d4aca45429094aa0fea349041. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:01:38.513734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-1d2f2644-a3f7-4621-96ba-b6d73c614578 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for user f78ab5496c1e4008b83fa3866601c06c and project 5b4d791d4aca45429094aa0fea349041. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:01:38.520062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:38.522825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:38.537396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:38.538374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:38.541826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:38.545864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:38.646571 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1d2f2644-a3f7-4621-96ba-b6d73c614578 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.594594 May 23 13:01:38.647086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 118/243] 10.222.0.30 () {62 vars in 1199 bytes} [Mon May 23 13:01:38 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 596 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 23 13:01:38.772261 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:38.782545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-62b42367-8992-4b73-b0a6-e24f033a1cc1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.284501 May 23 13:01:38.783461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 126/244] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:38 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:39.515767 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:39.518248 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:39.525095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:39.525752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:39.526796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:39.528375 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:39.793161 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:39.799354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:39.800580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:39.811637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-839d7ad5-80db-4209-984f-333872e93aa5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.304973 May 23 13:01:39.820786 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 119/245] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:39 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:39.827024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:39.827804 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:39.828805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:39.829393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:40.031097 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:40.047819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-54b41f29-8ad6-4432-a59b-6b416ca94f70 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.252339 May 23 13:01:40.049205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 127/246] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:39 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:40.827708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:40.828892 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:40.835643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:40.836457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:40.837428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:40.839463 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:41.071428 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:41.072184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:41.087964 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:41.088641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:41.090047 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:41.103027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:41.173580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:41.187216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7cee053e-d680-4198-a0c7-8ad47ddbf5ec tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.358965 May 23 13:01:41.203524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 120/247] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:40 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:41.382141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:41.392948 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f15b1f87-6637-495b-b19a-ebbe93a4b7de tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.325177 May 23 13:01:41.404941 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 128/248] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:41 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:41.695609 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-fe242506-6e71-499d-b89e-99d344b686d9 req-e67cbae7-b6c7-4c1e-a156-26048a5ce62c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8", "name": "network-vif-plugged", "status": "completed", "tag": "01e9d1a4-308e-43f5-9c0e-42aa73aab730"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:41.730964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-fe242506-6e71-499d-b89e-99d344b686d9 req-e67cbae7-b6c7-4c1e-a156-26048a5ce62c service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:41.730964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-fe242506-6e71-499d-b89e-99d344b686d9 req-e67cbae7-b6c7-4c1e-a156-26048a5ce62c service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:41.755530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-fe242506-6e71-499d-b89e-99d344b686d9 req-e67cbae7-b6c7-4c1e-a156-26048a5ce62c service nova] Creating event network-vif-plugged:01e9d1a4-308e-43f5-9c0e-42aa73aab730 for instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 on hv-cinder-maste May 23 13:01:41.778592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-fe242506-6e71-499d-b89e-99d344b686d9 req-e67cbae7-b6c7-4c1e-a156-26048a5ce62c service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092174 May 23 13:01:41.779457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 121/249] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:01:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:42.159619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-159335722", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "08745ac9-8019-40f9-9db5-aea6cd887576"}], "security_groups": [{"name": "tempest-securitygroup--2033597935"}], "key_name": "tempest-keypair-596350624", "user_data": "}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:42.207175 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:42.208962 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:42.218168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:42.218983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:42.221005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:42.221785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:42.428622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:42.439857 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3bcbd470-b8fd-4690-b5a2-9acc5a126844 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.237700 May 23 13:01:42.441792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 122/250] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:42 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:42.446626 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:42.447392 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:42.454398 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:42.454941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:42.456126 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:42.456570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:42.626201 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:42.634500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8d6a43b0-477d-4bfb-b395-f822da6d1c6e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.191439 May 23 13:01:42.635073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 123/251] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:42 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:43.462551 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:43.462551 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:43.464859 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:43.465318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:43.466122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:43.466555 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:43.606225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:43.614921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-04065c91-dedb-4f90-9a34-7ba8447abc0d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1314 microversion: 2.1 time: 0.161477 May 23 13:01:43.617130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 124/252] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:43 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:43.651707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:43.652267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:43.658259 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:43.658710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:43.668728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:43.668728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:43.801111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:43.808644 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8332f875-9326-43b8-a960-859d2286a428 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.160794 May 23 13:01:43.809216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 125/253] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:43 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:44.632012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:44.632706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:44.639787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:44.640342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:44.640991 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:44.641425 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:44.783700 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:44.791251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ad9a972f-15c2-4ace-b071-c1da05adfecd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1498 microversion: 2.1 time: 0.163148 May 23 13:01:44.791816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 126/254] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:44 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1498 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:44.826460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:44.829587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:44.833302 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:44.833749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:44.834591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:44.835027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:44.979215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:44.987840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e35e6f07-e4f7-4de0-8f25-db87839836da tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.166452 May 23 13:01:44.988468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 127/255] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:44 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:46.005658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:46.006332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:46.013051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:46.013493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:46.014543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:46.014972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:46.147158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:46.157238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fd7bef15-5dcd-4dcc-af43-eb3088831a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.155316 May 23 13:01:46.158176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 128/256] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:46 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:46.658374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:46.658864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:46.665575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:46.666063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:46.666836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:46.668504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:46.695899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:01:46.804741 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:47.050330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1dfdd6ec-0339-4c17-8df3-4bd967f4f968 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "DELETE /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 204 len: 0 microversion: 2.1 time: 0.395248 May 23 13:01:47.051018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 129/257] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:01:46 2022] DELETE /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:01:47.059567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:47.060259 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:47.066235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:47.066683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:47.067447 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:47.067883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:47.205479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:47.212875 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-15cd6ace-f57f-46a8-a4a8-67b3496a02bc tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.157784 May 23 13:01:47.213445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 130/258] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:47 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:47.219732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:47.220447 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:47.227121 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:47.227541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:47.235619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:47.235619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:47.376663 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:47.384319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-516b9298-50f1-4d9c-8a4d-3b595f21e4b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.168948 May 23 13:01:47.385101 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 131/259] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:47 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:48.230594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:48.232288 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:48.238775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:48.239221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:48.239986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:48.240417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:48.380339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:48.387884 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cd694868-ba44-4bc5-b0e3-fe902118af19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.161478 May 23 13:01:48.388448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 132/260] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:48 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:48.405574 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:48.406366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:48.415895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:48.415895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:48.415895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:48.415895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:48.525321 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=4 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:48.552321 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:48.560615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-93277d83-9409-49ae-a231-38c8ccc26e02 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.162531 May 23 13:01:48.564910 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 133/261] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:48 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:49.408320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:49.408928 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:49.415901 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:49.416258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:49.417011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:49.417441 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:49.556152 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:49.565072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-828537c2-9172-41c1-9758-6168db09d36e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.162574 May 23 13:01:49.565636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 134/262] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:49 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:49.581095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:49.581696 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:49.588215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:49.588517 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:49.589276 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:49.589706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:49.729115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:49.737905 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7983996a-0215-461a-a8a0-f57aeb4aedb7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.160534 May 23 13:01:49.738481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 135/263] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:49 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:50.042511 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] validate_networks() for [('08745ac9-8019-40f9-9db5-aea6cd887576', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:01:50.582739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:50.583509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:50.589858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:50.590413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:50.591098 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:50.591545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:50.604022 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:01:50.604452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:01:50.604916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:01:50.605319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:01:50.605733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:01:50.615005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:50.615449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:50.616242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:50.616675 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:50.639204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for project 5b4d791d4aca45429094aa0fea349041. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:01:50.643472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for user f78ab5496c1e4008b83fa3866601c06c and project 5b4d791d4aca45429094aa0fea349041. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:01:50.655962 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:01:50.658758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:01:50.733008 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:50.740495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-db086c0c-9c90-4135-8f2e-8691fd6f5220 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.162053 May 23 13:01:50.741063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 136/264] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:50 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:50.755371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:50.755986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:50.763655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:50.764095 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:50.765147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:50.765575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:50.773291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-06c4ad1b-7bd7-4aa7-b00f-6a91cfa21a23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 8.618439 May 23 13:01:50.779795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 129/265] 10.222.0.30 () {62 vars in 1187 bytes} [Mon May 23 13:01:42 2022] POST /compute/v2.1/servers => generated 402 bytes in 8620 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 23 13:01:50.785266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7a3a9554-5b3b-4484-96f0-dd827e6fc62f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:50.785895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7a3a9554-5b3b-4484-96f0-dd827e6fc62f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:50.864333 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a3a9554-5b3b-4484-96f0-dd827e6fc62f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:50.872809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a3a9554-5b3b-4484-96f0-dd827e6fc62f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:50.874218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a3a9554-5b3b-4484-96f0-dd827e6fc62f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:50.875318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7a3a9554-5b3b-4484-96f0-dd827e6fc62f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1280 microversion: 2.20 time: 0.094314 May 23 13:01:50.875870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 130/266] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:50 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1280 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:01:50.935690 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:50.943586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-78d620d1-a7ed-4c74-a4af-b22b32f6a68b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.192237 May 23 13:01:50.944149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 137/267] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:50 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:51.758646 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:51.759526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:51.767159 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:51.767603 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:51.768358 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:51.768785 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:51.894032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:51.894645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:51.900712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:51.901168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:51.901963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:51.902402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:51.903784 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:51.911427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-62e3aa56-cbc3-477b-89ec-c05f8808c8f7 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.156648 May 23 13:01:51.911993 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 131/268] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:51 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:51.960643 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:51.961281 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:51.963463 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:51.968517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:51.968967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:51.969728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:51.970193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:51.972541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:51.974078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:51.975183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4f6b14eb-eaa2-469b-bf49-239ef14b91d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1284 microversion: 2.20 time: 0.085838 May 23 13:01:51.975779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 138/269] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:51 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1284 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:01:52.107137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:52.114479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f4e3b478-7a58-4f86-a4ed-4e8f5cc64282 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.157775 May 23 13:01:52.115061 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 132/270] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:51 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:52.928317 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:52.928961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:52.930848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:52.936635 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:52.937078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:52.937831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:52.938304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:52.993828 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:52.994457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:53.001687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:53.002161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:53.002915 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:53.003339 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:53.090360 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:53.098253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:53.099761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:53.100845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ac82efca-8138-4fab-aec4-a553bb8a80a0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1284 microversion: 2.20 time: 0.111536 May 23 13:01:53.106384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 133/271] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:52 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1284 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:01:53.117007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:53.125081 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-56b4b4f0-2074-4d4a-a236-0f3ecc64e596 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.200456 May 23 13:01:53.125577 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 139/272] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:52 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:53.131466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:53.132100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:53.138902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:53.139347 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:53.140347 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:53.140778 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:53.269009 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:53.276436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3f01485c-869e-46d1-9870-7d76f63badbd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.148735 May 23 13:01:53.276993 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 134/273] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:53 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:54.118515 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:54.119123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:54.126361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:54.126840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:54.127596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:54.128030 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:54.142237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:54.142777 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:54.148984 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:54.149429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:54.150205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:54.150630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:54.192111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:54.200091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:54.201573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:54.202692 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2d48fdd1-ade5-4e22-8476-ce213b3a189a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1284 microversion: 2.20 time: 0.088067 May 23 13:01:54.203252 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 140/274] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:54 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:01:54.287942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:54.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:54.294622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:54.296103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a89eb59a-d789-4680-a5fe-8245e920d0c1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.158024 May 23 13:01:54.296679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 135/275] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:54 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:54.300745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:54.301189 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:54.301972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:54.302404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:54.436493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:54.443971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a7a91a4b-0126-4e82-a2a2-3c9c53bff9e4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.155428 May 23 13:01:54.444533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 141/276] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:54 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:55.221087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:55.221700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:55.228078 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:55.228521 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:55.229267 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:55.229691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:55.289379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:55.297912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:55.299383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:55.300455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c55eb0bc-efe3-4d58-8688-6fcecc58a386 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1284 microversion: 2.20 time: 0.083263 May 23 13:01:55.301000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 136/277] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:55 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1284 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:01:55.312551 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:55.313163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:55.319117 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:55.319563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:55.320350 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:55.320791 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:55.444735 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:55.452627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-613b12d8-a986-47cc-a257-0009f798d4df tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.144128 May 23 13:01:55.453178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 142/278] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:55 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:55.460263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:55.460877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:55.467032 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:55.467531 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:55.468286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:55.468754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:55.596150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:55.603433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-98398d4a-6007-44c8-99fe-be6cdf3263f8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.146946 May 23 13:01:55.604032 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 137/279] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:55 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:56.318475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:56.318988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:56.325096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:56.325573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:56.326352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:56.326788 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:56.386878 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:56.395219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:56.396689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:56.397759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3a7c35d3-69df-4758-b7cc-81320019f320 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1284 microversion: 2.20 time: 0.083307 May 23 13:01:56.398342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 143/280] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:56 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1284 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:01:56.469236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:56.469891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:56.476310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:56.476756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:56.477512 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:56.477970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:56.610534 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:56.618034 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2a8db5af-a17c-40ed-b505-028428b4bf43 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.152642 May 23 13:01:56.618602 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 138/281] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:56 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:56.620581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:56.621200 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:56.628098 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:56.628545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:56.629298 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:56.629734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:56.761988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:56.771003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-03b2d8b9-137b-4628-bcec-ed533709b8c0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.154093 May 23 13:01:56.771560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 144/282] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:56 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:01:57.415046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:57.415646 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:57.422292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:57.422732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:57.423478 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:57.423907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:57.441409 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:57.486172 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:57.494943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:57.496457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:57.497687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fba69a8b-cdf0-441b-b10a-08b03e9ee072 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1284 microversion: 2.20 time: 0.086526 May 23 13:01:57.498383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 139/283] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:57 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:01:57.635557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:57.636155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:57.642982 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:57.642982 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:57.643636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:57.644061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:57.776686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:57.785165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a4304a3a-8b26-40a3-8f92-60eff8af1272 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.153442 May 23 13:01:57.785920 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 145/284] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:57 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:57.787839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:57.788585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:57.794663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:57.795111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:57.795877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:57.796313 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:57.927538 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:57.935366 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7343b4f3-210e-4488-aba8-fe25565bf0f0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.151318 May 23 13:01:57.935926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 140/285] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:57 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:58.516889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:58.517701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:58.525035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:58.525549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:58.526529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:58.527023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:58.549144 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:58.600280 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:58.609110 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:58.611300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:58.611890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ee3ff386-8e67-467b-b45e-5ac9f4227f70 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1276 microversion: 2.20 time: 0.099923 May 23 13:01:58.612458 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 146/286] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:58 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1276 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:01:58.802895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:58.803505 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:58.805872 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=74,nova_api:INSERT=4 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:01:58.811728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:58.812168 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:58.812927 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:58.813355 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:58.961976 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:58.965753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:58.969769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:58.970606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:58.971415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:58.971857 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:59.089286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:59.102366 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2610e4d2-eaff-4bdf-9de7-a488f2f91499 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1504 microversion: 2.1 time: 0.303843 May 23 13:01:59.110294 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 141/287] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:58 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1504 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:59.169813 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:59.177983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a8b5e5a9-367f-4ec2-9660-873290b9548a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.229940 May 23 13:01:59.183538 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 147/288] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:01:58 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:01:59.303277 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-3b51d80e-a991-4c16-b63d-4562605312d0 req-ffd59837-1fb4-489c-9900-cfb95c1777e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8", "name": "network-vif-deleted", "tag": "01e9d1a4-308e-43f5-9c0e-42aa73aab730"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:01:59.316862 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-3b51d80e-a991-4c16-b63d-4562605312d0 req-ffd59837-1fb4-489c-9900-cfb95c1777e5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:59.317533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-3b51d80e-a991-4c16-b63d-4562605312d0 req-ffd59837-1fb4-489c-9900-cfb95c1777e5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:59.331406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-3b51d80e-a991-4c16-b63d-4562605312d0 req-ffd59837-1fb4-489c-9900-cfb95c1777e5 service nova] Creating event network-vif-deleted:01e9d1a4-308e-43f5-9c0e-42aa73aab730 for instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 on hv-cinder-maste May 23 13:01:59.338919 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-3b51d80e-a991-4c16-b63d-4562605312d0 req-ffd59837-1fb4-489c-9900-cfb95c1777e5 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041636 May 23 13:01:59.339966 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 142/289] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:01:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:01:59.630694 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:01:59.631495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:01:59.638221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:59.638718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:59.639623 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:01:59.640324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:01:59.710942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:59.720755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:59.722823 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:01:59.723675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-64eadd43-1f23-4ff7-96c5-403f10fa3d6b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1332 microversion: 2.20 time: 0.096825 May 23 13:01:59.724301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 148/290] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:01:59 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1332 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:00.127912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:00.128603 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:00.134715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:00.135156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:00.135901 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:00.136328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:00.198870 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:00.200062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:00.205599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:00.206580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:00.207124 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:00.207886 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:00.208374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:00.213332 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6f33daf4-7fcd-4e8f-962c-d580c65f25d3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1462 microversion: 2.1 time: 0.089607 May 23 13:02:00.213948 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 143/291] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:00 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1462 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:00.342452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:00.358056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c8c01c30-f1dc-4479-9343-5d5679c6f525 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.156165 May 23 13:02:00.358056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 149/292] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:00 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:00.742578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:00.743447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:00.746523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:00.752141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:00.752593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:00.753366 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:00.753798 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:00.817686 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:00.826817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:00.828970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:00.830951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3101d00f-d1b5-40fb-8ee5-26b04d04adca tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1332 microversion: 2.20 time: 0.093239 May 23 13:02:00.831986 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 144/293] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:00 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1332 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:01.231756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:01.233035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:01.239628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.240294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.241145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.241685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.281249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.281734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.282225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lazy-loading 'fault' on Instance uuid f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:02:01.327593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:01.336219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-53ea9f44-a814-45f0-ae1b-52bad9e028c0 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1304 microversion: 2.1 time: 0.109003 May 23 13:02:01.337043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 150/294] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:01 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1304 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:01.413471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:01.413471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:01.413471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=8 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:01.413471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.414200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.414200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.414200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.550159 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:01.557569 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-716d1221-7211-4093-8b05-0a41dddfe2fb tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.193136 May 23 13:02:01.558167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 145/295] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:01 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:01.848587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:01.849204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:01.855137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.855576 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.857192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:01.857628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:01.919337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:01.927272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:01.928756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:01.929830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-83a7383f-80c8-42cf-8e80-3863f3038a29 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1340 microversion: 2.20 time: 0.085240 May 23 13:02:01.930416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 151/296] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:01 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1340 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:02.430323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:02.430945 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:02.440205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.440665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:02.441450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.441906 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:02.475337 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.475773 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:02.476218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lazy-loading 'fault' on Instance uuid f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:02:02.516748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:02.524138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f1de65b0-dd04-4d4e-ba6c-66b8c5b13dff tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 200 len: 1304 microversion: 2.1 time: 0.097656 May 23 13:02:02.524707 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 146/297] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:02 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 1304 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:02.575540 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:02.576135 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:02.582217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.582658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:02.583412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.583840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:02.711158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:02.718647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3c47caa6-729a-4714-807a-004af62e0ea6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.147557 May 23 13:02:02.719244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 152/298] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:02 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:02.948714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:02.949312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:02.956260 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.956705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:02.957780 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:02.958206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:03.089025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:03.097757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:03.099406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:03.100617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-13577fa7-4232-4c0f-9e1f-843b3e4d5019 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1350 microversion: 2.20 time: 0.156272 May 23 13:02:03.101160 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 147/299] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:02 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1350 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:03.541949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:03.542765 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:03.550055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:03.550611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:03.551457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:03.551967 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:03.566880 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] HTTP exception thrown: Instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 could not be found. May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Request method failure captured: May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 HTTP/1.1 May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4WdHBODb2l-qfs70RHIAaPJA5cwQcrqALRjbzxM8WO0lc6gKPMlH6P_0r_DSkiV9AxGKFYAUaObqKcEM0i--bFATksWw07GkR9aSE_z1Alem5nFCzNguhdnVaoeTH0JFFaECuhQSIBzWQrH3urtEvr8iYQVcqRVd8JqS8iJNP__9IE May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 3b3719ab772449d79dc00dc96487dc36 May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesActionsTest-784262700 May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:02:03.568659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesActionsTest-784262700 May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 3b3719ab772449d79dc00dc96487dc36 May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesActionsTest-784262700 May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesActionsTest-784262700-project May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 7f5e411f4d8e4f13b662cf2846720cd3 May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesActionsTest-784262700-project May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 could not be found. May 23 13:02:03.570495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8'} May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 could not be found. May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:02:03.571319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 could not be found. May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 could not be found. May 23 13:02:03.572527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:02:03.573589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Returning 404 to user: Instance f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:02:03.573589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c430b525-aa39-4edf-8484-d606963cf1b3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8" status: 404 len: 111 microversion: 2.1 time: 0.031941 May 23 13:02:03.573589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 153/300] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:03 2022] GET /compute/v2.1/servers/f3d43155-ac13-4a2b-bf73-ddb2d75bb7a8 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:02:03.678537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-2134780622", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "networks": [{"uuid": "58c39c65-d0c2-47c3-84c7-3d9e660a4739"}]}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:03.737000 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:03.742420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:03.751929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:03.752547 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:03.753351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:03.753842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:03.757118 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] validate_networks() for [('58c39c65-d0c2-47c3-84c7-3d9e660a4739', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:02:04.022554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:04.031200 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-92d77a23-31a7-4b7c-ad66-60868bb523be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1521 microversion: 2.1 time: 0.298531 May 23 13:02:04.031773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 154/301] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:03 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1521 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:04.119473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:04.120088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:04.126900 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:04.127927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:04.128313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:04.128818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:04.699938 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:04.706048 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:04.707894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:04.709682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3860ab3f-f297-43ed-a3c4-78571c5ea915 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.593738 May 23 13:02:04.710476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 155/302] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:04 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:02:04.896651 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:02:04.897172 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:02:04.897551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:02:04.898026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:02:04.899343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:02:04.909365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:04.909949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:04.910804 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:04.911433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:04.928679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Getting quotas for project 3b3719ab772449d79dc00dc96487dc36. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:02:04.933250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Getting quotas for user 7f5e411f4d8e4f13b662cf2846720cd3 and project 3b3719ab772449d79dc00dc96487dc36. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:02:04.948145 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:02:04.951140 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:02:05.048623 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:05.049139 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:05.055153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:05.055606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:05.056359 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:05.056795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:05.115713 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e1efc58e-64e1-428a-b1a2-a1250cbf2893 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.441014 May 23 13:02:05.122227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 148/303] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:02:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1448 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:02:05.128875 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-868228f7-fa40-43d1-a6ad-1e19b58ba04c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:05.129932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-868228f7-fa40-43d1-a6ad-1e19b58ba04c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:05.291859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-868228f7-fa40-43d1-a6ad-1e19b58ba04c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:05.300601 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-868228f7-fa40-43d1-a6ad-1e19b58ba04c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1214 microversion: 2.1 time: 0.177001 May 23 13:02:05.310043 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 149/304] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:05 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1214 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:05.428054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:05.440351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-406ec6e5-c340-47da-92ef-75342be9c39a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1501 microversion: 2.1 time: 0.395540 May 23 13:02:05.441451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 156/305] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:05 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1501 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:05.462510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:05.463369 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:05.470573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:05.471282 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:05.472236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:05.473553 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:05.730010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:05.730201 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:05.730201 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:05.739183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5afeee92-50fa-4a45-a2be-82409bcb5d99 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1516 microversion: 2.1 time: 0.280337 May 23 13:02:05.740825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 150/306] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:05 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1516 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:05.741582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:05.742069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:05.742854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:05.743287 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:05.968282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:05.976473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:05.977975 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:05.979129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1c527652-8e2f-46da-be90-9ed69fce6275 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.254054 May 23 13:02:05.979676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 157/307] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:05 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:06.098682 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-b437369d-1eb5-402c-8060-33f016e2a206 req-d1184c27-75c4-4993-b7d7-06c0ed76e389 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:06.109036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-b437369d-1eb5-402c-8060-33f016e2a206 req-d1184c27-75c4-4993-b7d7-06c0ed76e389 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:06.109485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-b437369d-1eb5-402c-8060-33f016e2a206 req-d1184c27-75c4-4993-b7d7-06c0ed76e389 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:06.128776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-b437369d-1eb5-402c-8060-33f016e2a206 req-d1184c27-75c4-4993-b7d7-06c0ed76e389 service nova] Creating event network-changed:cee5aa36-f69a-432b-9f71-0cc93809d9a9 for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 on hv-cinder-maste May 23 13:02:06.134226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-b437369d-1eb5-402c-8060-33f016e2a206 req-d1184c27-75c4-4993-b7d7-06c0ed76e389 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038960 May 23 13:02:06.135312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 151/308] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:06.317839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:06.318438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:06.324315 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:06.324760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:06.330813 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:06.330813 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:06.410054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:06.418170 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-df8da8f8-9752-4af3-8235-46e7c0673f19 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1218 microversion: 2.1 time: 0.103923 May 23 13:02:06.419572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 158/309] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:06 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1218 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:06.998690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:06.999411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:07.006159 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:07.006878 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:07.007658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:07.008095 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:07.194590 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:07.203623 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:07.205115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:07.206243 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a6c9b92f-8e30-44ca-95f1-a8ca7eb6a731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.212517 May 23 13:02:07.206874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 152/310] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:06 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:07.435850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:07.439580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:07.442452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:07.442905 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:07.443669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:07.444107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:07.511901 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:07.530640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6a714cbc-3b6b-492a-97c2-45045494babb tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1266 microversion: 2.1 time: 0.098884 May 23 13:02:07.534825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 159/311] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:07 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1266 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:08.225090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:08.225722 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:08.235133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:08.235861 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:08.236670 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:08.237096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:08.366563 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:08.375022 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:08.376498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:08.377574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cc9f92a4-98d6-4e9a-8d87-513cf0dd75e0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.157043 May 23 13:02:08.378183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 153/312] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:08 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:08.549587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:08.550335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:08.556637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:08.557084 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:08.558106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:08.558290 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:08.627750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:08.637118 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-279ac86b-5f05-41b5-be82-109a34e26675 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1274 microversion: 2.1 time: 0.091300 May 23 13:02:08.638440 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 160/313] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:08 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1274 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:08.723571 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:08.724518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:08.730406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:08.733541 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:08.733541 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:08.733541 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:08.755165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lazy-loading 'flavor' on Instance uuid 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:02:09.081512 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-66da14e4-d2ea-4613-b60f-4ccba13af6b6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/action" status: 200 len: 32 microversion: 2.1 time: 0.361841 May 23 13:02:09.083087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 154/314] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:02:08 2022] POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/action => generated 32 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:02:09.091904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:09.092523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:09.098451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:09.098899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:09.099742 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:09.100178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:09.232620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:09.240261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-67984c80-c92b-4f2a-9d11-c7b039a28c78 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.152422 May 23 13:02:09.240832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 161/315] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:09 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:09.397221 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:09.398038 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:09.404886 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:09.405301 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:09.406093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:09.406573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:09.536891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:09.545266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:09.546878 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:09.547967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ee6d8f2d-4519-4c23-a09b-9a02b804f68f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.155595 May 23 13:02:09.548514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 155/316] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:09 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:09.659219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:09.659812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:09.667557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:09.667557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:09.667557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:09.667557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:09.760899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:09.771013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2dd78443-67a3-4d45-bcdc-c6f402b8d824 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1274 microversion: 2.1 time: 0.115875 May 23 13:02:09.772011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 162/317] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:09 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1274 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:10.258981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:10.259721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:10.266422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:10.266972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:10.267849 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:10.268486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:10.413472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:10.417077 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:10.422127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b40b1f2a-9e06-415a-b940-18c651ad7f76 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.167559 May 23 13:02:10.429501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 156/318] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:10 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:10.566556 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:10.567165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:10.573227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:10.573672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:10.574499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:10.575665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:10.594593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:10.727208 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:10.736574 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:10.738115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:10.739419 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5e014ed3-0bb2-4a7a-ba15-d2677c75cb8f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.176986 May 23 13:02:10.739983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 163/319] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:10 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:10.787718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:10.788322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:10.795585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:10.796070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:10.796782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:10.797217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:10.866054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:10.874120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-04e946f6-d608-48e5-9265-0604d662095c tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1274 microversion: 2.1 time: 0.090250 May 23 13:02:10.874761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 157/320] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:10 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1274 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:11.439548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:11.440150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:11.446146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:11.446587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:11.447343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:11.447771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:11.660187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:11.669870 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2885f10a-009b-4ceb-873f-3b3bf6ebae32 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.234149 May 23 13:02:11.670645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 164/321] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:11 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:11.758246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:11.763669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:11.768766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:11.768766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:11.768766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:11.768766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:11.894777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:11.896217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:11.914705 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:11.915166 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:11.916391 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:11.916990 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:12.041084 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:12.058342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8df095c3-901a-4bcb-bf7e-3aae8c3e3089 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1274 microversion: 2.1 time: 0.170538 May 23 13:02:12.059836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 165/322] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:11 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1274 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:12.089354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:12.100532 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:12.103489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:12.110212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-26d71f1b-90ab-41cb-8aba-dae8b868aaad tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.351801 May 23 13:02:12.110212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 158/323] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:11 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:12.689185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:12.694289 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:12.703167 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:12.704393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:12.705212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:12.705831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:13.055794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1088163237", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1848200419", "security_groups": [{"name": "tempest-secgroup-smoke-267520630"}], "block_device_mapping_v2": [{"uuid": "5f90287d-4df4-48dc-8bb0-9c8de261c2fd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b8997ea7-0013-4a19-8df5-a24d1eca94b5"}]}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:13.135372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:13.148731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-df1ac33a-142c-4623-8332-99b09863525c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.463302 May 23 13:02:13.149328 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 166/324] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:12 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:13.163481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:13.166468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:13.175057 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:13.175689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:13.176862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:13.177528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:13.290659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:13.298741 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-70e2aa64-9b6c-4765-bd76-b79888130dac tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1272 microversion: 2.1 time: 0.143422 May 23 13:02:13.299442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 167/325] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:13 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1272 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:13.315492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:13.319466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:13.332105 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:13.332899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:13.335790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] validate_networks() for [('b8997ea7-0013-4a19-8df5-a24d1eca94b5', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:02:13.350242 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:13.350718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:13.612549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:13.622709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:13.624438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:13.625729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-62b7a02a-5936-4ec4-be1d-91a9f3f7c3b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.324987 May 23 13:02:13.626651 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 168/326] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:13 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:13.972104 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:02:13.972610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:02:13.973090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:02:13.973631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:02:13.974072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:02:13.982959 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:13.983438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:13.984237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:13.986714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.001626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:02:14.009799 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for user c6a6096760734ac49ce2f26c6065b701 and project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:02:14.087887 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:02:14.090068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='5f90287d-4df4-48dc-8bb0-9c8de261c2fd',volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:02:14.167967 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:14.169560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:14.178331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:14.178795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.180006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:14.181258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.430839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:14.443009 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-05c6f418-2c5d-489c-b6e3-5ed438e225b8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.280206 May 23 13:02:14.445821 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 169/327] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:14 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:14.457633 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:14.458273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:14.470345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:14.471035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.471993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:14.472643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.558781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-89b877f6-2909-49e7-a026-70acb2ed7a18 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.507546 May 23 13:02:14.560695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 159/328] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:02:13 2022] POST /compute/v2.1/servers => generated 405 bytes in 1510 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:02:14.570364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c7b0b104-4b4a-4831-8b4b-6463a35b9cea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:14.571109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c7b0b104-4b4a-4831-8b4b-6463a35b9cea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:14.670799 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c7b0b104-4b4a-4831-8b4b-6463a35b9cea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:14.682685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:14.687510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c7b0b104-4b4a-4831-8b4b-6463a35b9cea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1095 microversion: 2.1 time: 0.111542 May 23 13:02:14.688341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 160/329] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:14 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1095 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:14.697564 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f5007b3f-f78a-4976-bff1-59d25a16f245 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.249020 May 23 13:02:14.701135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:14.701502 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 170/330] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:14 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:14.702127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:14.708871 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:14.709730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.710905 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:14.711486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:14.915129 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:14.925347 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:14.938048 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:14.939794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6a62f5e7-dd76-4700-9404-6e60ce8a3c81 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.250328 May 23 13:02:14.940572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 161/331] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:14 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:15.106109 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-774f88f0-036a-4171-a6bc-156a19bcc05a req-6705cffc-7343-47e1-9105-a2432eca7d0b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "edb18d57-e69d-4723-910e-8ba6fb05d761", "tag": "772c5e46-6e5d-4eee-b775-bb97851e12d1"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:15.116706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-774f88f0-036a-4171-a6bc-156a19bcc05a req-6705cffc-7343-47e1-9105-a2432eca7d0b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.117487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-774f88f0-036a-4171-a6bc-156a19bcc05a req-6705cffc-7343-47e1-9105-a2432eca7d0b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.130656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-774f88f0-036a-4171-a6bc-156a19bcc05a req-6705cffc-7343-47e1-9105-a2432eca7d0b service nova] Creating event network-changed:772c5e46-6e5d-4eee-b775-bb97851e12d1 for instance edb18d57-e69d-4723-910e-8ba6fb05d761 on hv-cinder-maste May 23 13:02:15.135625 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-774f88f0-036a-4171-a6bc-156a19bcc05a req-6705cffc-7343-47e1-9105-a2432eca7d0b service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033551 May 23 13:02:15.142382 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 171/332] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:15.461168 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:15.461650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:15.471628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.474346 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.475396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.475832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.626827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-56ba06c3-08c0-46ba-802e-c965b3248193 req-dc848d3a-a064-4767-87a9-3cc9699f4a47 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "name": "network-vif-plugged", "status": "completed", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:15.642722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-56ba06c3-08c0-46ba-802e-c965b3248193 req-dc848d3a-a064-4767-87a9-3cc9699f4a47 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.643520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-56ba06c3-08c0-46ba-802e-c965b3248193 req-dc848d3a-a064-4767-87a9-3cc9699f4a47 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.655642 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-56ba06c3-08c0-46ba-802e-c965b3248193 req-dc848d3a-a064-4767-87a9-3cc9699f4a47 service nova] Creating event network-vif-plugged:cee5aa36-f69a-432b-9f71-0cc93809d9a9 for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 on hv-cinder-maste May 23 13:02:15.660502 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-56ba06c3-08c0-46ba-802e-c965b3248193 req-dc848d3a-a064-4767-87a9-3cc9699f4a47 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042788 May 23 13:02:15.661232 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 172/333] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:02:15.697714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:15.710990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-62e41878-6540-42c9-baef-ddb015dbf104 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.253940 May 23 13:02:15.724781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 162/334] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:15 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:15.729580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:15.730232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:15.731989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:15.732759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:15.740871 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.742918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.746758 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.749424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.750260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.760484 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.760484 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.760484 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.951814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:15.974253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-78f1067e-9e71-4606-9391-d2ed32ffa396 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1145 microversion: 2.1 time: 0.270272 May 23 13:02:15.979096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 173/335] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:15 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1145 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:15.981065 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:15.981931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:15.989554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.990312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:15.991316 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:15.992141 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:16.174865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:16.186245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e2a810a4-9517-47b3-ae1f-709f5028db16 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.460259 May 23 13:02:16.187027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 163/336] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:15 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:16.382616 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:16.391445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:16.393062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:16.394217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-80ced001-9f51-41cf-b7ce-c94ca9ff3b8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.418377 May 23 13:02:16.394918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 174/337] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:15 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:16.728501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:16.729098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:16.735761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:16.736197 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:16.736944 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:16.737371 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:16.991354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:16.992066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:16.999698 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:17.000215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:17.001057 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:17.001561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:17.108100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:17.116135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a714bcef-ffc7-4fc6-9dfa-461e7ed05494 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.391454 May 23 13:02:17.117070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 164/338] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:16 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:17.123921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:17.131990 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f910ed1e-82cf-43ee-9d96-550a74ab2312 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.144487 May 23 13:02:17.132582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 175/339] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:16 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:17.203767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:17.204377 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:17.210270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:17.210756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:17.211529 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:17.211971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:17.347790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:17.355102 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-94621d89-3377-471a-a47d-e6768101011d tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.155777 May 23 13:02:17.355594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 165/340] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:17 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:17.412557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:17.413301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:17.419924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:17.420378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:17.421137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:17.421571 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:17.554692 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:17.564217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:17.565736 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:17.566893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1f7cf903-461b-4a6d-93d1-99163102db63 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.158311 May 23 13:02:17.567475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 176/341] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:17 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:18.133764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:18.134716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:18.142218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.142672 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.147398 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.147398 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.149436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:18.150075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:18.156688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.157075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.157839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.158331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.238408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:18.245872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-db6380f7-86d8-49e5-8d66-42a3ccd686c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.100091 May 23 13:02:18.246476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 177/342] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:18 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:18.295903 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:18.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-38c72d48-0ce5-4223-bf41-91e26981530b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.173721 May 23 13:02:18.304130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 166/343] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:18 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:18.373369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:18.374069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:18.379943 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.380388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.381147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.381578 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.514184 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:18.521844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9feb85c3-f24b-4d95-a59b-7697fd4448c9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.153291 May 23 13:02:18.522439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 178/344] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:18 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:18.585338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:18.585969 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:18.592341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.592788 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.593550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:18.594011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:18.722982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:18.731627 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:18.733133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:18.734259 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2661d764-7611-4c45-9652-d4ce713d5a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.153045 May 23 13:02:18.738309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 167/345] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:18 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:19.039865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-700048943", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "networks": [{"uuid": "f040d182-eca3-4e6d-946e-e964e751bb04"}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:19.091493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] validate_networks() for [('f040d182-eca3-4e6d-946e-e964e751bb04', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:02:19.263236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:19.263851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:19.270821 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.271276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.272051 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.272494 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.336649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.344739 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-94c6767b-c944-4a51-a835-581a81256066 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.085415 May 23 13:02:19.353918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 168/346] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:19 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:19.353918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:19.353918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:19.356936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.357534 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.358336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.358774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.519664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:02:19.520171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:02:19.520544 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:02:19.520954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:02:19.521398 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:02:19.530974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.531425 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.532227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.532667 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.543576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.550334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Getting quotas for project ccd27933078b44b48323995e98b21318. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:02:19.551141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cabf7345-690e-4eaa-a669-a55e7127d054 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.204892 May 23 13:02:19.551880 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 169/347] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:19 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:19.554336 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Getting quotas for user 5c6147fed25a4212b6175aacc460943d and project ccd27933078b44b48323995e98b21318. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:02:19.556216 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:19.556809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:19.562387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.562834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.563584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.564012 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.565516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:02:19.567803 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:02:19.683311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fca41d9f-b327-4be9-97ea-fb15b1f94a89 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.647240 May 23 13:02:19.684367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 179/348] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:02:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 649 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:02:19.694296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-362fc8e6-c6f7-44d4-8ab6-0acb1e9d3d31 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:19.695019 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-362fc8e6-c6f7-44d4-8ab6-0acb1e9d3d31 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:19.695384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.704039 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:19.704755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d675dcde-2d5b-4bdd-ba0f-4be951cd94cf tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.151753 May 23 13:02:19.705288 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 170/349] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:19 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:19.752201 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:19.752801 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:19.753469 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-362fc8e6-c6f7-44d4-8ab6-0acb1e9d3d31 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.759209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.759797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.760551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:19.760654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-362fc8e6-c6f7-44d4-8ab6-0acb1e9d3d31 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1218 microversion: 2.1 time: 0.070278 May 23 13:02:19.760983 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:19.761758 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 180/350] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:19 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1218 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:19.904973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.913180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.914652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:19.915760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-625c3e4f-9bad-464f-a2ac-959e0055fc35 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.167460 May 23 13:02:19.916344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 171/351] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:19 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:02:20.361496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:20.362185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:20.371653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.371653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.371653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.371653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.435161 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:20.442806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0ff39cdd-2997-4af1-9718-56ee05c9b51a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.085539 May 23 13:02:20.443388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 181/352] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:20 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:20.568511 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:20.569111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:20.576433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.576881 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.577632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.578100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.712872 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:20.720429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5557bca2-cc57-4aa3-8521-5c7ebe9003ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.155787 May 23 13:02:20.720990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 172/353] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:20 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:20.725190 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:20.725190 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:20.727944 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.728381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.729130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.729560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.779702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:20.780353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:20.782300 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:20.787019 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.787556 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.788322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.788757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.861551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:20.869663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-71aa8d6a-bc60-41b6-a457-221d55c9c062 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1222 microversion: 2.1 time: 0.094508 May 23 13:02:20.870344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 173/354] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:20 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1222 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:20.885338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:20.893433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bf12e20f-c1c5-472e-8d54-601cefef484a tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.175592 May 23 13:02:20.895565 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 182/355] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:20 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:20.934589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:20.935196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:20.941218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.941666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:20.942457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:20.942892 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.082803 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:21.090623 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:21.092107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:21.093188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c7a732bc-ebbc-4743-9e4e-312dea29d0a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.162687 May 23 13:02:21.093735 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 174/356] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:20 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:21.459002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:21.459603 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:21.466050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.466496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.467239 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.467668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.471511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:21.529698 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:21.539497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9505606d-060e-44f1-a8be-958f56f26658 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.084202 May 23 13:02:21.540069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 183/357] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:21 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:21.737567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:21.739128 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:21.745828 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.746357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.747322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.747749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.752776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:21.877715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:21.885166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7df1ccee-a654-4053-a036-1c0f08995d54 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.151541 May 23 13:02:21.885726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 175/358] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:21 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:21.886726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:21.887893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:21.894192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.894638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.895397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.895830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.910406 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:21.911020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:21.916936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.917369 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.918174 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:21.918630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:21.964585 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:21.972939 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-23123d9d-07d7-4ed3-ad57-707b39876ddb tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1222 microversion: 2.1 time: 0.089956 May 23 13:02:21.973524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 184/359] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:21 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1222 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:22.063280 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:22.070656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2e8d20bc-4743-45ae-b561-7ff09d022438 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.164151 May 23 13:02:22.071214 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 176/360] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:21 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:22.111043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:22.111648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:22.117509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.117983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.118741 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.119181 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.246308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:22.254675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:22.256156 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:22.257249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e64bfcd8-3403-4e37-8126-92208fee1e3f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.150969 May 23 13:02:22.257804 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 185/361] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:22 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:22.557549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:22.558109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:22.564840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.565286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.566071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.566508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.634098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:22.641946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6ee67b11-156c-4b1e-a140-263faa1b54ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.088251 May 23 13:02:22.642507 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 177/362] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:22 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:22.903415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:22.904382 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:22.910708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.911155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.911921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.912356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.990431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:22.991100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:22.997746 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.998293 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:22.999122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:22.999657 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.054615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.062180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-537630aa-54fd-47e6-9e71-fb859a9a1db7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.163185 May 23 13:02:23.062754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 186/363] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:22 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:23.070582 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.077838 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b67316be-4849-477b-884d-078d5057a25b tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1222 microversion: 2.1 time: 0.091378 May 23 13:02:23.078420 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 178/364] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:22 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1222 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:23.088420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:23.089032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:23.095743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:23.096194 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.096961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:23.097399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.229673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.237622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-356f9faa-375c-4b0d-b049-30aa4bcfd8e9 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.153157 May 23 13:02:23.238206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 187/365] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:23 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:23.275504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:23.276107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:23.283262 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:23.283707 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.284455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:23.284887 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.410641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.418864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.420319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.421418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b5ec6169-44eb-4d07-b225-5426cb890609 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.149810 May 23 13:02:23.422334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 179/366] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:23 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:23.658567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:23.659165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:23.665171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:23.665551 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.666327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:23.666755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:23.731754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:23.739414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-22dc2224-3a05-4704-9b2e-4d8b962ae896 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.084686 May 23 13:02:23.740294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 188/367] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:23 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:24.078656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:24.079230 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:24.080978 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=4 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:24.086217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.086666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.087423 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.087858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.095716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:24.096317 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:24.102242 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.102687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.103447 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.103883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.182209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.189573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dfc0decf-2fe7-4dcb-a544-ca2d4b19c7b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1222 microversion: 2.1 time: 0.097622 May 23 13:02:24.190172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 189/368] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:24 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1222 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:24.248679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.254435 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:24.255079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:24.256182 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cfaceedc-eefa-4d84-ad98-26a029f0bcf7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.181990 May 23 13:02:24.256752 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 180/369] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:24 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:24.261049 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.261497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.263397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.263852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.394934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.402255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-07abe3f3-2116-4136-866b-4691cd1c05f2 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.151694 May 23 13:02:24.403324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 190/370] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:24 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:24.439871 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:24.440482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:24.446451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.447001 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.447794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.448227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.584664 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.592868 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.594376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.595461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fabbf7f2-4fb2-46d5-9dd9-be2a1e5323f9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1406 microversion: 2.20 time: 0.159553 May 23 13:02:24.596121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 181/371] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:24 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1406 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:02:24.756994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:24.757599 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:24.763576 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.764018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.764772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:24.765212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:24.826041 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:24.833435 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-69d6934e-f4db-434d-8469-896db5702355 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1193 microversion: 2.1 time: 0.080355 May 23 13:02:24.834029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 191/372] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:24 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1193 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:25.206786 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:25.207386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:25.210178 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=16 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:25.214146 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.214593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.216275 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.218419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.274198 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:25.274771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:25.283573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.284261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.284399 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.285132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.285620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.300698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f28c02a3-fae9-4f40-8889-bff9df195c74 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1222 microversion: 2.1 time: 0.097591 May 23 13:02:25.302133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 182/373] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:25 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1222 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:25.417109 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.420356 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:25.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:25.425267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b69e527f-9adc-4b44-ae24-405f3df82e97 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.155039 May 23 13:02:25.425835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 192/374] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:25 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:25.426957 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.427465 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.428224 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.428655 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.556806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.564263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-65aae080-039f-4fd5-95a1-e4f14135b386 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.148659 May 23 13:02:25.564815 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 183/375] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:25 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:25.613703 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:25.614342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:25.620519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.620961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.621902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.622343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.754460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.762877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.764430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.765561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5b776b80-9182-40bb-a394-984a4946606b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1598 microversion: 2.20 time: 0.155736 May 23 13:02:25.766399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 193/376] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:02:25 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:02:25.784940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:25.785458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:25.791457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.791901 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.792700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.793129 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.853735 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:25.854461 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:25.861596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.862067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.862830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:25.863258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:25.877533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-caa0af93-5f72-44b3-8b08-013445a06176 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.097258 May 23 13:02:25.878953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 184/377] 10.222.0.30 () {60 vars in 1315 bytes} [Mon May 23 13:02:25 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-interface => generated 284 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:02:25.931876 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:25.941832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-700e503c-4f08-491c-a883-2b4c70db3196 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1201 microversion: 2.1 time: 0.094953 May 23 13:02:25.943021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 194/378] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:25 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1201 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:26.191889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-5091cf24-bb2d-431e-b86b-c5d1cf674801 req-9fda6745-0068-4953-8209-3f6702ee85cc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "edb18d57-e69d-4723-910e-8ba6fb05d761", "name": "network-vif-plugged", "status": "completed", "tag": "772c5e46-6e5d-4eee-b775-bb97851e12d1"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:26.201693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5091cf24-bb2d-431e-b86b-c5d1cf674801 req-9fda6745-0068-4953-8209-3f6702ee85cc service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.202156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5091cf24-bb2d-431e-b86b-c5d1cf674801 req-9fda6745-0068-4953-8209-3f6702ee85cc service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.215084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-5091cf24-bb2d-431e-b86b-c5d1cf674801 req-9fda6745-0068-4953-8209-3f6702ee85cc service nova] Creating event network-vif-plugged:772c5e46-6e5d-4eee-b775-bb97851e12d1 for instance edb18d57-e69d-4723-910e-8ba6fb05d761 on hv-cinder-maste May 23 13:02:26.222342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-5091cf24-bb2d-431e-b86b-c5d1cf674801 req-9fda6745-0068-4953-8209-3f6702ee85cc service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035228 May 23 13:02:26.223514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 185/379] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:26.319050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:26.319885 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:26.335922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.336169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.337204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.337797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.429711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:26.443891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:26.445448 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:26.453169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6108f351-4481-4d40-9a3c-6d24bb848738 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1214 microversion: 2.1 time: 0.138831 May 23 13:02:26.465924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 195/380] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:26 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1214 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:26.466187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.466187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.466187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.466187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.585193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:26.586377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:26.606349 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.606868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.611454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.611932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.739751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:26.753010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8f94ab64-cb65-457f-8a9d-e54cacb9890b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.313966 May 23 13:02:26.763720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 186/381] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:26 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:26.963550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:26.964787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:26.966614 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:26.972936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.973605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.974935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:26.975516 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:26.975784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d4db13dc-41e7-49d6-92e9-a824fac66684 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.397372 May 23 13:02:26.986046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 196/382] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:26 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:27.081296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:27.089031 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-194e0aee-f8f5-4072-8663-0ce1256189f3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1211 microversion: 2.1 time: 0.131770 May 23 13:02:27.089701 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 187/383] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:26 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1211 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:27.476188 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:27.476797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:27.482870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-71dd0d72-d746-4a71-9e1c-60ad3e2b3384 req-565d3a3c-4058-42b0-b5ea-e213ca5c0bdd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:27.494764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-71dd0d72-d746-4a71-9e1c-60ad3e2b3384 req-565d3a3c-4058-42b0-b5ea-e213ca5c0bdd service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:27.495167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-71dd0d72-d746-4a71-9e1c-60ad3e2b3384 req-565d3a3c-4058-42b0-b5ea-e213ca5c0bdd service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:27.499666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:27.500323 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:27.501918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:27.501918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:27.521693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-71dd0d72-d746-4a71-9e1c-60ad3e2b3384 req-565d3a3c-4058-42b0-b5ea-e213ca5c0bdd service nova] Creating event network-changed:cee5aa36-f69a-432b-9f71-0cc93809d9a9 for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 on hv-cinder-maste May 23 13:02:27.526827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-71dd0d72-d746-4a71-9e1c-60ad3e2b3384 req-565d3a3c-4058-42b0-b5ea-e213ca5c0bdd service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047573 May 23 13:02:27.527443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 188/384] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:27.587183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:27.595923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-233633d6-936c-4505-992e-a0a3f30616c3 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1278 microversion: 2.1 time: 0.123430 May 23 13:02:27.597363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 197/385] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:27 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1278 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:27.771449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:27.772145 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:27.778377 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:27.778832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:27.779708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:27.780157 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:27.992709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:27.993673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:28.001561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:28.002259 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:28.003044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:28.003639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:28.004482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:28.015162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-86c5d2d4-078d-4dee-90e7-613e9c1c1f8e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.247799 May 23 13:02:28.025644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 189/386] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:27 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:28.108246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:28.109146 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:28.116216 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:28.117088 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:28.117663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:28.118228 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:28.306681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:28.315074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d69b1063-12f3-45e9-a7c1-7c58281224a5 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1314 microversion: 2.1 time: 0.326318 May 23 13:02:28.315740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 198/387] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:27 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1314 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:28.373192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:28.380812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0c5ef90e-7873-4b6c-b410-3c9abd39aa72 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1278 microversion: 2.1 time: 0.276648 May 23 13:02:28.381382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 190/388] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:28 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1278 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:28.614420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:28.615362 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:28.621763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:28.622260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:28.623128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:28.623637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:28.935816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:28.941211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:28.946443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6c543d12-5b73-4986-993f-42a9f0ad416e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1278 microversion: 2.1 time: 0.335894 May 23 13:02:28.947357 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 199/389] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:28 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1278 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:29.034570 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:29.035232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:29.043001 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.044066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.045003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.046138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.260896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:29.269151 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ccc4a10b-7b52-48ba-b0ba-22b052a966a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.239785 May 23 13:02:29.269715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 191/390] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:29 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:29.331949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:29.332605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:29.343197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.343665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.344484 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.344920 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.401614 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:29.401614 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:29.408279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.409121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.410134 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.410634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.541454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:29.758376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=8 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:29.832477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:29.857563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0f0473f0-7dc5-4145-b170-597f1d077e15 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1498 microversion: 2.1 time: 0.525860 May 23 13:02:29.859732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 200/391] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:29 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1498 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:29.875987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-258e5670-a55b-4ed9-ab4a-88f58edb791a req-9c7e8f56-f345-4db4-b5d6-f20e1bd4b6a1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7064ed49-237d-441d-a087-70b195245624", "tag": "f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:29.892701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-258e5670-a55b-4ed9-ab4a-88f58edb791a req-9c7e8f56-f345-4db4-b5d6-f20e1bd4b6a1 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.893489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-258e5670-a55b-4ed9-ab4a-88f58edb791a req-9c7e8f56-f345-4db4-b5d6-f20e1bd4b6a1 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.907955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-258e5670-a55b-4ed9-ab4a-88f58edb791a req-9c7e8f56-f345-4db4-b5d6-f20e1bd4b6a1 service nova] Creating event network-changed:f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25 for instance 7064ed49-237d-441d-a087-70b195245624 on hv-cinder-maste May 23 13:02:29.913626 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-258e5670-a55b-4ed9-ab4a-88f58edb791a req-9c7e8f56-f345-4db4-b5d6-f20e1bd4b6a1 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052464 May 23 13:02:29.914888 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 201/392] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:29.919457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:29.928094 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0aeb4b19-c5ce-4e2f-b3fc-f01f3e0a3357 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1278 microversion: 2.1 time: 0.533063 May 23 13:02:29.928765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 192/393] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:29 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1278 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:29.964971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:29.965735 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:29.972951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.973473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:29.974429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:29.974858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:30.097984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:30.105499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-78446a47-31a7-4468-a2c5-0847bb18fde5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1276 microversion: 2.1 time: 0.144773 May 23 13:02:30.106908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 202/394] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:29 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1276 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:30.291464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:30.291464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:30.294590 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:30.295270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:30.296212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:30.296716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:30.316316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:30.672911 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:30.681099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a5466484-2179-4946-a49b-32872e1c315f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.398449 May 23 13:02:30.681684 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 193/395] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:30 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:30.946200 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:30.946792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:30.953900 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:30.954392 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:30.955163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:30.955596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:31.123530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:31.124135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:31.130169 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:31.131982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:31.131982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:31.131982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:31.164701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:31.173072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ac22b9cf-23de-4f5e-bbd4-5f940393d553 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1278 microversion: 2.1 time: 0.230813 May 23 13:02:31.173645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 203/396] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:30 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1278 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:31.261483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:31.269693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b8f8e2f7-f69b-4523-89ad-da864cb2a4e1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.150107 May 23 13:02:31.270316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 194/397] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:31 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:31.700302 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:31.701064 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:31.707072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:31.707516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:31.708267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:31.708698 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:31.864310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:31.872933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0b36d798-d57e-45af-b8dd-c93d73e837a8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.177287 May 23 13:02:31.873598 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 204/398] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:31 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:32.202946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:32.203748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:32.210969 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.211403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.212291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.212691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.296325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:32.296325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:32.296325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.296325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.297256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.297256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.847078 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:32.858943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3ecee813-4924-4418-884a-8bf3b8ac0dab tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1278 microversion: 2.1 time: 0.672389 May 23 13:02:32.859623 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 195/399] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:32 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1278 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:32.864454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:32.865025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:32.870985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.871616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.872489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.873017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.899699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:32.901554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:02:32.909267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c9d2ab44-50fb-4c21-90c7-147d90eb8c11 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.625632 May 23 13:02:32.912912 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 205/400] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:32 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:32.917638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:32.924042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:32.931481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.936034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:32.938545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:32.939062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.172252 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:33.179791 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-868ae99b-7bc7-4594-a6aa-4afd5ab6177a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.265659 May 23 13:02:33.180409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 206/401] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:32 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:33.256235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ffa4fff3-e1e3-4a99-8082-fb0128106826 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "DELETE /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 204 len: 0 microversion: 2.1 time: 0.395141 May 23 13:02:33.256826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 196/402] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:02:32 2022] DELETE /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:02:33.265895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:33.266862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:33.272894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.273334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.274118 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.274545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.499996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:33.507324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-150df865-fe65-4ec0-b7b5-6ed08a8944b1 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.246197 May 23 13:02:33.508163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 207/403] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:33 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:33.719192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-88777cae-9982-468d-bfab-657ed6d92e5e req-4f5ecada-96c6-4d6a-a82f-7721a4a28fe7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4cae571e-2b00-4426-a0e8-aa2c62e48617", "tag": "6e114242-9ed0-40d0-98db-97f57599dd87"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:33.727371 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-88777cae-9982-468d-bfab-657ed6d92e5e req-4f5ecada-96c6-4d6a-a82f-7721a4a28fe7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.727808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-88777cae-9982-468d-bfab-657ed6d92e5e req-4f5ecada-96c6-4d6a-a82f-7721a4a28fe7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.740154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-88777cae-9982-468d-bfab-657ed6d92e5e req-4f5ecada-96c6-4d6a-a82f-7721a4a28fe7 service nova] Creating event network-changed:6e114242-9ed0-40d0-98db-97f57599dd87 for instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 on hv-cinder-maste May 23 13:02:33.744676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-88777cae-9982-468d-bfab-657ed6d92e5e req-4f5ecada-96c6-4d6a-a82f-7721a4a28fe7 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029543 May 23 13:02:33.745251 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 197/404] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:33.875725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:33.876327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:33.882308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.882755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.883504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.883937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.930706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:33.931397 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:33.938170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.938620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:33.939374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:33.939806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:34.060934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:34.068531 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6ab08852-8dff-489b-bdb5-4c9c2f661bcc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.197301 May 23 13:02:34.069172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 208/405] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:33 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:34.095966 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:34.103173 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-724ee4f3-3aeb-4fdb-8750-abeff962f93a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.176695 May 23 13:02:34.103733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 198/406] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:33 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:34.197305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:34.197929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:34.203807 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:34.204249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:34.204999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:34.205431 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:34.335703 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:34.343664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bed931ec-051d-49d0-a6bd-339c5e3af7ae tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.150103 May 23 13:02:34.346204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 209/407] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:34 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:34.524498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:34.525105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:34.532232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:34.532700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:34.533455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:34.533971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:34.661968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:34.669998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2c8be311-52e7-47f5-9946-bcd65423fa20 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.149289 May 23 13:02:34.670588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 199/408] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:34 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:35.085468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:35.086139 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:35.092135 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.092574 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.093322 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.093751 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.120393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:35.121087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:35.128258 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.128634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.129394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.129826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.280526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:35.287820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1ecdf4d3-8e9a-40b5-9ff6-e21943720189 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.206165 May 23 13:02:35.288384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 210/409] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:35 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:35.301814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:35.309679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b8d092a2-4d2a-4a3b-a238-8de7ece42277 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.192449 May 23 13:02:35.310373 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 200/410] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:35 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:35.360761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:35.361366 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:35.367278 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.367725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.368481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.368915 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.493688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:35.501593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f876f7c0-aeb6-49a3-a64f-db70422a7e4a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.144607 May 23 13:02:35.502205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 211/411] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:35 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:35.687042 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:35.687641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:35.694091 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.694592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.695295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:35.695721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:35.831506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:35.838907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c32f1ae2-f092-4071-9d35-d6cc24f6cf92 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.155759 May 23 13:02:35.839468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 201/412] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:35 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:36.304887 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:36.305482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:36.312648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.313090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.313842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.314301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.326444 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:36.327670 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:36.333545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.334027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.334809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.335242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.515024 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:36.522123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:36.522665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bcf20bbe-ad2b-4e9d-9f31-bb199a675fd0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.221632 May 23 13:02:36.523230 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 212/413] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:36 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:36.527543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:36.528640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:36.531026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d20181bd-cf09-4619-8418-96984aaab75e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.208289 May 23 13:02:36.531774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 202/414] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:36 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:36.534390 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.534833 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.535587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.536015 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.671734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:36.679079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a12b8b5f-c890-4376-addd-e10f5a9ffc5d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.154926 May 23 13:02:36.679636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 213/415] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:36 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:36.855850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:36.856553 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:36.863135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.863624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:36.864391 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:36.864862 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.011319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:37.019530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2db8f08a-fcee-43a1-9cda-bcd1087449c3 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.167699 May 23 13:02:37.020082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 203/416] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:36 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:37.539637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:37.540240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:37.547253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:37.547694 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.548443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:37.548553 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:37.548873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.549054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:37.554935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:37.555383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.556142 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:37.556576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.742090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:37.748986 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:37.749805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6fbd7dab-dd0a-45eb-b878-3c47cb23d623 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.213904 May 23 13:02:37.750405 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 214/417] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:37 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:37.754825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:37.755430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:37.756715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-afdcce83-f5be-4914-83e6-5324092ef255 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.212017 May 23 13:02:37.757377 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 204/418] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:37 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:37.761352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:37.761802 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.763404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:37.764186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:37.896957 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:37.904456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-275e5321-cffa-4a50-bc37-ab449292718c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.153106 May 23 13:02:37.905013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 215/419] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:37 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:38.036115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:38.036747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:38.043709 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:38.044152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:38.045256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:38.045687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:38.175741 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:38.183190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7abfdeaa-63c3-4dfb-9e92-609dce2adcbd tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.150973 May 23 13:02:38.183762 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 205/420] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:38 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:38.767521 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:38.768162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:38.774432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:38.774825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:38.775595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:38.776033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:38.779096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:38.779730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:38.785668 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:38.791829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:38.791829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:38.791829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:38.979988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:38.984227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:38.989537 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c3258295-2fe6-4bcd-b8dc-a575620acbb4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.226050 May 23 13:02:38.990242 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 216/421] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:38 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:38.991910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-457a0803-85c7-4d87-afff-780be99f1871 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.216891 May 23 13:02:38.992645 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 206/422] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:38 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:38.995215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:38.995828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:39.002583 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.003064 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.003831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.004269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.008434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:39.151405 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:39.158920 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2c1e816e-f37c-468e-832b-41448edf977d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1502 microversion: 2.1 time: 0.167690 May 23 13:02:39.159486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 217/423] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:38 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1502 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:39.181951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b8ea6028-11cf-45e7-a2a7-02fd2b2d17a4"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:39.183109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:39.190473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.190924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.191685 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.192123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.200815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:39.201496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:39.207741 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.208197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.208967 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.209407 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.215663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] HTTP exception thrown: Cannot 'attach_volume' instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c while it is in vm_state rescued May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Request method failure captured: May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/os-volume_attachments HTTP/1.1 May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 74 May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VVZ6qL1baoXQCGJ4rKxwU3PdgDazbVRgPzsvNfj6Z6e7pYM7EagYaFpYssjuDdpTcMmzVLV3_9gBjZFaKnL-a51HATs8ndSjnf_Aso8WExsxhXHAAytDHLZmW36WO0T3MDIwJfdqsUibcr3o-dkGgwCQnPJhHaHKfTcLTweehZiVM May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:02:39.217450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: b18e5f1b24634096995068e41d94cad6 May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {"volumeAttachment": {"volumeId": "b8ea6028-11cf-45e7-a2a7-02fd2b2d17a4"}} May 23 13:02:39.219059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Cannot 'attach_volume' instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c while it is in vm_state rescued May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'server_id': '3d4562fc-a49b-4d0d-b486-e8d2fb59db7c', 'body': {'volumeAttachment': {'volumeId': 'b8ea6028-11cf-45e7-a2a7-02fd2b2d17a4'}}} May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c in vm_state rescued. Cannot attach_volume while the instance is in this state. May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:02:39.219812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c while it is in vm_state rescued May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:02:39.221050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c while it is in vm_state rescued May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Returning 409 to user: Cannot 'attach_volume' instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c while it is in vm_state rescued {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2250c324-801f-42ce-bf3b-398e31d10c37 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.041086 May 23 13:02:39.222226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 207/424] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:02:39 2022] POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/os-volume_attachments => generated 152 bytes in 42 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 23 13:02:39.228019 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:39.228554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:39.235706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.236150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.237038 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.237474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.289072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:39.347149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:39.354681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a41194d5-7f7f-4b4c-8aa6-6309f21e6ee8 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.157903 May 23 13:02:39.358021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 218/425] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:39 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:39.434675 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dcdee0c4-4917-49c3-a489-eb71d1e4e963 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/action" status: 202 len: 0 microversion: 2.1 time: 0.210451 May 23 13:02:39.436273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 208/426] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:02:39 2022] POST /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c/action => generated 0 bytes in 212 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:02:39.443300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:39.443907 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:39.450149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.451301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.451301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:39.451515 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:39.584050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:39.593281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b1d8d85f-d8dc-41fa-8827-361a56714dfe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.152467 May 23 13:02:39.593281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 219/427] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:39 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:40.007754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:40.008497 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:40.009762 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:40.010411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:40.015861 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.018558 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.018663 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.018663 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.018848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.018848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.019002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.019002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.220926 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:40.224561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:40.228502 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cd8906d6-7ab1-4f7b-bc91-d5533b89c1ea tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.222584 May 23 13:02:40.229073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 220/428] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:40 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:40.231991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-077ef62f-00bb-42e0-8c46-fc17091772a5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.228460 May 23 13:02:40.232504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 209/429] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:40 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:40.372828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:40.373498 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:40.379463 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.379941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.380710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.381129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.514869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:40.522686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8c9b985e-0508-4375-8ebf-1a8d74ba0d12 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1504 microversion: 2.1 time: 0.153857 May 23 13:02:40.523330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 221/430] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:40 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1504 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:40.608569 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:40.609254 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:40.615204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.615770 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.616662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:40.617103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:40.820713 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:40.829899 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f883ff38-1023-4e4d-b9b5-8394948363a3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.225266 May 23 13:02:40.839460 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 210/431] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:40 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:41.252014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:41.253164 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:41.253718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:41.254341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:41.267833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.269612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.271618 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.271618 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.271618 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.274608 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.275566 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.281116 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.703417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:41.708540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:41.716320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d51f04e2-5508-411b-95d4-7e40c700ae4a tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.474431 May 23 13:02:41.722926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ff6202a9-ca98-443d-9c95-94cfc4ffa347 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.478188 May 23 13:02:41.724619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 211/433] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:41 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:41.724825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 222/433] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:41 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:41.733674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-041e990c-880c-486f-996c-f949d54cbc13 req-457ceb4e-debb-4073-9b50-559edceb60a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "edb18d57-e69d-4723-910e-8ba6fb05d761", "name": "network-vif-deleted", "tag": "772c5e46-6e5d-4eee-b775-bb97851e12d1"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:41.739583 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:41.742131 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:41.760472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.763695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.765937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-041e990c-880c-486f-996c-f949d54cbc13 req-457ceb4e-debb-4073-9b50-559edceb60a1 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.767351 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.767964 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.768570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-041e990c-880c-486f-996c-f949d54cbc13 req-457ceb4e-debb-4073-9b50-559edceb60a1 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.797651 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-041e990c-880c-486f-996c-f949d54cbc13 req-457ceb4e-debb-4073-9b50-559edceb60a1 service nova] Creating event network-vif-deleted:772c5e46-6e5d-4eee-b775-bb97851e12d1 for instance edb18d57-e69d-4723-910e-8ba6fb05d761 on hv-cinder-maste May 23 13:02:41.819810 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-041e990c-880c-486f-996c-f949d54cbc13 req-457ceb4e-debb-4073-9b50-559edceb60a1 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092320 May 23 13:02:41.820719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 223/434] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:41.847509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:41.848192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:41.855461 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.855908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.859426 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:41.864480 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:41.864673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:41.875353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d0eeea22-469c-4cea-8de7-e78c3b428880 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1299 microversion: 2.1 time: 0.148239 May 23 13:02:41.875353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 212/435] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:41 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1299 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:42.087831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:42.096079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9b5547b1-cc5c-493b-8f39-7aefc915e61b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.252697 May 23 13:02:42.096643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 224/436] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:41 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:42.733383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:42.734038 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:42.740783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:42.740965 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:42.741404 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:42.741571 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:42.742030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:42.742466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:42.749908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:42.750372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:42.751134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:42.751561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:42.952648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:42.956650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:42.960147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a6732cac-d88a-4b15-b5f1-e4aaf949bd02 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.223432 May 23 13:02:42.960707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 225/437] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:42 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:42.964778 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-08c75111-7065-4b84-81a0-55ba271b7897 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.235988 May 23 13:02:42.965516 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 213/438] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:42 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:42.965662 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:42.966262 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:42.972016 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:42.972455 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:42.973290 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:42.973715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.004600 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.005041 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.005471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lazy-loading 'fault' on Instance uuid edb18d57-e69d-4723-910e-8ba6fb05d761 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:02:43.055533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:43.063411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f1f99b63-6841-4d85-a179-5e1f1b62484e tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1304 microversion: 2.1 time: 0.101434 May 23 13:02:43.069397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 226/439] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:42 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1304 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:43.113317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:43.113982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:43.121911 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.122955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.123638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.124198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.485090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:43.492955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-76a3e0fc-04c4-4050-9c88-c536f49cfd1c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.383557 May 23 13:02:43.493506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 214/440] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:43 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:43.977367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:43.978278 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:43.981973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:43.982444 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:43.985348 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.985944 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.986810 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.987337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.988706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.989153 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:43.989920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:43.990456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:44.194256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:44.201744 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:44.202915 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d91bfa09-ad51-4c4f-952a-68c339b1e59f tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.229623 May 23 13:02:44.203560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 227/441] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:43 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:44.209234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8adadcc1-291e-498d-beb0-a4fe97203e4e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.231361 May 23 13:02:44.209978 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 215/442] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:43 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:44.211465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:44.212129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:44.218889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:44.219342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:44.220114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:44.220560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:44.247453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:44.250991 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:44.251442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:44.251891 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lazy-loading 'fault' on Instance uuid edb18d57-e69d-4723-910e-8ba6fb05d761 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:02:44.294353 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f5e411f4d8e4f13b662cf2846720cd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b3719ab772449d79dc00dc96487dc36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:44.301972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1f65da53-f711-4215-a3ee-1e1105a32724 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 200 len: 1304 microversion: 2.1 time: 0.094240 May 23 13:02:44.302729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 228/443] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:44 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 1304 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:44.509430 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:44.510297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:44.517329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:44.518400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:44.518400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:44.518980 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:44.650392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:44.657968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2b2505ba-4750-4190-aeca-91f004224d19 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.152351 May 23 13:02:44.658532 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 216/444] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:44 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:45.219248 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:45.220035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:45.226408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.230519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.230519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.230519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.230779 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:45.230779 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:45.237101 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.237745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.238564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.238998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.437163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:45.442310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:45.456065 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7a19d5f5-d535-459e-a916-5b7bc8a5903f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.231837 May 23 13:02:45.456647 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 217/445] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:45 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:45.460404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8401fb1a-9af9-41b5-938f-3100c1a9d3d1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.245092 May 23 13:02:45.461053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 229/446] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:45 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:45.461266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:45.461863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] [instance: edb18d57-e69d-4723-910e-8ba6fb05d761] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:45.470459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.470838 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.472639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.473080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.487739 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] HTTP exception thrown: Instance edb18d57-e69d-4723-910e-8ba6fb05d761 could not be found. May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Request method failure captured: May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 HTTP/1.1 May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WdHBODb2l-qfs70RHIAaPJA5cwQcrqALRjbzxM8WO0lc6gKPMlH6P_0r_DSkiV9AxGKFYAUaObqKcEM0i--bFATksWw07GkR9aSE_z1Alem5nFCzNguhdnVaoeTH0JFFaECuhQSIBzWQrH3urtEvr8iYQVcqRVd8JqS8iJNP__9IE May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 3b3719ab772449d79dc00dc96487dc36 May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesActionsTest-784262700 May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:02:45.489379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/3b3719ab772449d79dc00dc96487dc36"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesActionsTest-784262700 May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 3b3719ab772449d79dc00dc96487dc36 May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesActionsTest-784262700 May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesActionsTest-784262700-project May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 7f5e411f4d8e4f13b662cf2846720cd3 May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesActionsTest-784262700-project May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance edb18d57-e69d-4723-910e-8ba6fb05d761 could not be found. May 23 13:02:45.490972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'edb18d57-e69d-4723-910e-8ba6fb05d761'} May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance edb18d57-e69d-4723-910e-8ba6fb05d761 could not be found. May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:02:45.491757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance edb18d57-e69d-4723-910e-8ba6fb05d761 could not be found. May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance edb18d57-e69d-4723-910e-8ba6fb05d761 could not be found. May 23 13:02:45.492942 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:02:45.494015 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] Returning 404 to user: Instance edb18d57-e69d-4723-910e-8ba6fb05d761 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:02:45.494015 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8686c102-b470-4c32-ac16-1fb49bcd6711 tempest-VolumesActionsTest-784262700 tempest-VolumesActionsTest-784262700-project] 10.222.0.30 "GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761" status: 404 len: 111 microversion: 2.1 time: 0.033199 May 23 13:02:45.494015 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 218/447] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:45 2022] GET /compute/v2.1/servers/edb18d57-e69d-4723-910e-8ba6fb05d761 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:02:45.674699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:45.675315 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:45.684222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.684746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.686189 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:45.686717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:45.832076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:45.837539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-98e2f717-9c24-4b10-adf2-5ba9ea4e283f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.166629 May 23 13:02:45.848352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 230/448] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:45 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:46.473149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:46.474003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:46.479811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:46.489475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:46.489683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:46.489683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:46.489683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:46.489683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:46.490046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:46.490046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:46.490046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:46.490046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:46.729178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:46.738207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9ef334bf-0618-4887-b026-8c1e771e6eae tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.262089 May 23 13:02:46.745313 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:46.748775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 231/449] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:46 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:46.753845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cecd2f71-63e1-470b-9d2b-b168d7dc2dee tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.284487 May 23 13:02:46.754444 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 219/450] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:46 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:46.855291 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:46.855958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:46.862294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:46.865584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:46.865584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:46.865584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:47.034853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:47.044191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ea50f313-715e-4860-9ad6-9c45b8ca39a5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.193070 May 23 13:02:47.045377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 232/451] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:46 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:47.756461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:47.757073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:47.759692 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:47.768626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:47.768626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:47.768626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:47.768626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:47.773613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:47.774358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:47.783452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:47.784110 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:47.785430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:47.791369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:47.974561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:47.982703 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-464fa1e1-39b3-4fc9-93d3-6aa5e86d1305 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.229351 May 23 13:02:47.983327 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 220/452] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:47 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:47.989233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:47.992773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:47.997046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b97b75ac-e13d-41cf-9814-a23e74a2091e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.230290 May 23 13:02:47.998238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 233/453] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:47 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:48.068560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:48.069208 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:48.075123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:48.075607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:48.076457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:48.076829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:48.212388 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:48.220625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-83e6ae81-8e94-4563-b810-17670d0b5699 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.156106 May 23 13:02:48.221491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 221/454] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:48 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:48.999993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:49.000606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:49.007324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:49.007772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:49.008705 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:49.009145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:49.017439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:49.018076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:49.024385 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:49.024829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:49.025756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:49.026371 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:49.215720 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:49.220361 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:49.223795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-416198ad-dedd-4d8e-9093-86ab998d24f2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.228223 May 23 13:02:49.224367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 234/455] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:48 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:49.228373 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8d6cb9c4-46f6-4738-80e1-98bfea91210d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.215770 May 23 13:02:49.229020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 222/456] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:49 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:49.237026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:49.237632 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:49.243619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:49.244069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:49.244831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:49.245323 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:49.382134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:49.390414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-34094259-a559-4f00-9aea-5e7a591c6e17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.157283 May 23 13:02:49.390971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 235/457] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:49 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:50.240667 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:50.241390 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:50.245021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:50.245643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:50.248235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:50.248554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:50.249335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:50.249774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:50.251924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:50.252371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:50.253124 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:50.255497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:50.460108 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:50.462520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:50.462679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a797a983-6ceb-4cdf-8a3f-7a65143cf266 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.226004 May 23 13:02:50.463192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 223/458] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:50 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:50.469216 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:50.470363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:50.470826 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-520acfd7-434b-4c28-875a-e49699f8b68e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.229444 May 23 13:02:50.479327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 236/459] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:50 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:50.486342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:50.486797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:50.487561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:50.487992 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:50.616847 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:50.625316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4c94406e-3a3f-4fd1-8f2c-a5f03bc9a7b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.161036 May 23 13:02:50.625902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 224/460] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:50 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:51.479696 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:51.480295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:51.490497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:51.490616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:51.491022 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:51.491106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:51.491183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:51.491183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:51.494790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:51.495242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:51.496006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:51.496440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:51.687273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:51.692136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:51.695415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-15928c0a-ba15-4f2a-b27a-b6db7b02d775 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.219394 May 23 13:02:51.695706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:02:51.696572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 237/461] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:51 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:51.700628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fa592741-bd8a-4dbf-8d79-5e339369a417 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.217026 May 23 13:02:51.701142 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:51.701917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 225/462] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:51 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:51.702090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:51.708143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:51.708588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:51.712856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:51.712856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:51.841496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:51.848987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0ac7af30-4f74-4da9-9edf-c7f70c29d3c7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.151553 May 23 13:02:51.849756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 238/463] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:51 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:52.254171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-ef8b92f0-9ffc-41cb-a084-1e3adbd32f78 req-d91aa859-fd6f-48af-85da-59db2b761c42 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4cae571e-2b00-4426-a0e8-aa2c62e48617", "name": "network-vif-plugged", "status": "completed", "tag": "6e114242-9ed0-40d0-98db-97f57599dd87"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:52.263000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-ef8b92f0-9ffc-41cb-a084-1e3adbd32f78 req-d91aa859-fd6f-48af-85da-59db2b761c42 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.263552 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-ef8b92f0-9ffc-41cb-a084-1e3adbd32f78 req-d91aa859-fd6f-48af-85da-59db2b761c42 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:52.276111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-ef8b92f0-9ffc-41cb-a084-1e3adbd32f78 req-d91aa859-fd6f-48af-85da-59db2b761c42 service nova] Creating event network-vif-plugged:6e114242-9ed0-40d0-98db-97f57599dd87 for instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 on hv-cinder-maste May 23 13:02:52.281592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-ef8b92f0-9ffc-41cb-a084-1e3adbd32f78 req-d91aa859-fd6f-48af-85da-59db2b761c42 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031073 May 23 13:02:52.282189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 226/464] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:52.712414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:52.713019 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:52.719615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:52.720300 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:52.720404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.720774 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:52.721535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.722001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:52.726809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.727255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:52.728031 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.728457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:52.919352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:52.923910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:52.928673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4de4a64a-8ace-4017-a857-71e52238b48d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.219958 May 23 13:02:52.929403 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 239/465] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:52 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:52.932275 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9ff2d5b2-df78-4ba5-9e06-0b705485ad93 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.218173 May 23 13:02:52.934517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 227/466] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:52 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:52.936880 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:52.937712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:52.947835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.947835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:52.947835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:52.947835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:53.085552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:53.094060 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e017ced4-9548-4e2b-b2cc-7977197db8be tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1510 microversion: 2.1 time: 0.163460 May 23 13:02:53.095579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 240/467] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:52 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1510 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:53.946308 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:53.946926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:53.954233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:53.954690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:53.955467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:53.955908 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:53.956378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:53.956984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:53.963522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:53.964054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:53.964978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:53.966806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:54.189120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:54.203222 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0148ad34-a791-48cf-884c-f5d08dac8852 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.260610 May 23 13:02:54.203989 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 228/468] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:53 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:54.207601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:54.218680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f08bebbb-c35e-4111-890c-9190601c7ed0 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.266233 May 23 13:02:54.225350 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:54.226229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:54.236379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:54.236766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:54.237550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:54.238265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:54.240817 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 241/469] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:53 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:54.455519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:54.466118 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b0e8e0d2-e6a6-4614-8a69-6bf2082a421d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1516 microversion: 2.1 time: 0.258846 May 23 13:02:54.478974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 229/470] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:54 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1516 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:55.228653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:55.238674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:55.239308 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:55.245643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:55.258112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:55.258533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:55.259309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:55.267370 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:55.269964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:55.271056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:55.272064 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:55.274757 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:55.573206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:55.584229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a33e73ce-f31f-462b-b9c1-05fb1c2df586 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1318 microversion: 2.1 time: 0.349290 May 23 13:02:55.587099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 230/471] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:55 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1318 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:55.612728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:55.642137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a2cc9873-0b46-46aa-8f4f-f3952d6e2ae6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.425409 May 23 13:02:55.645542 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 242/472] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:55 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:56.607580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:56.608791 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:56.616454 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:56.617137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:56.618132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:56.618573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:56.659954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:56.660672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:56.667090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:56.667535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:56.668293 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:56.668721 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:56.813525 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:56.824625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a62859ce-1946-448e-b7c9-6528606d6259 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1507 microversion: 2.1 time: 0.221462 May 23 13:02:56.825561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 231/473] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:56 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1507 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:56.887699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:56.900717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ce22592b-1191-4842-b61d-7c302bb437bb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.243661 May 23 13:02:56.901007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 243/474] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:56 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:02:57.501459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e694fd85-2778-4879-adf5-6b8ab38324ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'create', calling method: .version_select at 0x7f9b12f66790>, body: {"keypair": {"name": "tempest-keypair-1174189262", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLGBa2zIgBfAnIpgR9iNAryguhXwsYXcZ12Wwl2xdith/Uh9ye1HG6AtnOZhyT0ROlcg1oGKsxkieZROiBbjxC3KNIL2GemvovbBokC9KAWkm3JzFRtstiJ7me2PHNaexg=="}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:57.549164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-e694fd85-2778-4879-adf5-6b8ab38324ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:02:57.553451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-e694fd85-2778-4879-adf5-6b8ab38324ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for user b18e5f1b24634096995068e41d94cad6 and project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:02:57.699752 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e694fd85-2778-4879-adf5-6b8ab38324ac tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.202479 May 23 13:02:57.702802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 232/475] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:02:57 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:57.930941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:57.932273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:57.939631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:57.940374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:57.941267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:57.941900 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:58.193584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:58.205981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e050ef9f-f86f-4967-bce4-a75a92913557 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.287046 May 23 13:02:58.206628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 244/476] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:57 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:02:58.983490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-857b1f8a-ada1-4f34-a424-8d80f9d932ed req-bc662600-4b93-423c-8874-c6c6487e7552 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7064ed49-237d-441d-a087-70b195245624", "name": "network-vif-plugged", "status": "completed", "tag": "f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:02:59.017736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-857b1f8a-ada1-4f34-a424-8d80f9d932ed req-bc662600-4b93-423c-8874-c6c6487e7552 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:59.023679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-857b1f8a-ada1-4f34-a424-8d80f9d932ed req-bc662600-4b93-423c-8874-c6c6487e7552 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:59.053791 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-857b1f8a-ada1-4f34-a424-8d80f9d932ed req-bc662600-4b93-423c-8874-c6c6487e7552 service nova] Creating event network-vif-plugged:f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25 for instance 7064ed49-237d-441d-a087-70b195245624 on hv-cinder-maste May 23 13:02:59.063484 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-857b1f8a-ada1-4f34-a424-8d80f9d932ed req-bc662600-4b93-423c-8874-c6c6487e7552 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085572 May 23 13:02:59.064083 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 233/477] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:02:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:02:59.225202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:02:59.232460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:02:59.235861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:59.236268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:59.237200 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:02:59.237576 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:02:59.412030 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:02:59.435789 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1aab9010-54b8-4f69-b84a-d66d5442836b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.214971 May 23 13:02:59.447104 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 245/478] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:02:59 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:00.455014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:00.455788 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:00.469767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:00.470751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:00.471732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:00.472702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:00.719262 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:00.734952 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fae18403-a0bb-457f-9a51-aece63cfb9b3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.284302 May 23 13:03:00.740748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 234/479] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:00 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:01.305854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-242288763", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "11cfb087-2166-41ce-9a36-a5b349287bd2"}], "security_groups": [{"name": "tempest-securitygroup--989037791"}], "key_name": "tempest-keypair-1174189262", "user_data": "}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:01.758643 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:01.760160 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:01.768007 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:01.768461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:01.769239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:01.769784 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:01.936453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:01.943985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ead0fb80-da64-4fad-bd7b-953c63dbded8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1266 microversion: 2.1 time: 0.190271 May 23 13:03:01.944757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 235/480] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:01 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1266 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:02.964716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:02.965915 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:02.974678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:02.975405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:02.976420 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:02.977530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:03.136232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:03.147658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a0f0c450-fa2f-4512-ad9a-3f7c900f0917 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1453 microversion: 2.1 time: 0.188617 May 23 13:03:03.148234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 236/481] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:02 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1453 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:03.164620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:03.165232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:03.171518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:03.172199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:03.172974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:03.173574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:03.333476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:03.346708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f12bb84d-a4e6-4a39-8efc-d58da8cd054a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1453 microversion: 2.1 time: 0.185256 May 23 13:03:03.347543 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 237/482] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:03 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1453 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:06.012126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-8c588e91-bf5d-484a-a54d-6dd21fa0f005 req-d23dc080-8a08-4c73-b6e3-8b651dc968f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7064ed49-237d-441d-a087-70b195245624", "tag": "f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:06.021764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8c588e91-bf5d-484a-a54d-6dd21fa0f005 req-d23dc080-8a08-4c73-b6e3-8b651dc968f3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:06.026808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8c588e91-bf5d-484a-a54d-6dd21fa0f005 req-d23dc080-8a08-4c73-b6e3-8b651dc968f3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:06.035144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-8c588e91-bf5d-484a-a54d-6dd21fa0f005 req-d23dc080-8a08-4c73-b6e3-8b651dc968f3 service nova] Creating event network-changed:f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25 for instance 7064ed49-237d-441d-a087-70b195245624 on hv-cinder-maste May 23 13:03:06.040398 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-8c588e91-bf5d-484a-a54d-6dd21fa0f005 req-d23dc080-8a08-4c73-b6e3-8b651dc968f3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031982 May 23 13:03:06.041043 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 238/483] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:03:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:03:09.417231 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=72 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:11.323666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=37 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:15.600826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:15.601417 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:15.603912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=4 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:15.609808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:15.610490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:15.611300 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:15.611744 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:15.644658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-db1b13e0-82ba-4485-89c0-02e7bf5294a0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/ips" status: 200 len: 150 microversion: 2.1 time: 0.048015 May 23 13:03:15.645800 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 239/484] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:03:15 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/ips => generated 150 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:03:18.658188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "34a6495d-3741-497d-bb5d-d7dd465d95ff"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:18.659355 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:18.662993 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:18.667144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:18.668580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:18.668580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:18.668836 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:19.471981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-06ddbcbb-4e44-44f6-874c-cbe4218ca68b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.818502 May 23 13:03:19.478523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 240/485] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:03:18 2022] POST /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments => generated 194 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:23.024190 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] validate_networks() for [('11cfb087-2166-41ce-9a36-a5b349287bd2', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:03:23.475257 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:03:23.475745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:03:23.476312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:03:23.476786 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:03:23.477255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:03:23.487620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:23.488019 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:23.488826 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:23.489265 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:23.520920 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:03:23.525011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Getting quotas for user b18e5f1b24634096995068e41d94cad6 and project 5bda5ee012d84d88a3015baa3b80a7d7. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:03:23.536873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:03:23.539214 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:03:23.663823 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-948d7e5f-9486-458d-a3ee-1b70a8976d15 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 22.362205 May 23 13:03:23.664456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 246/486] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:03:01 2022] POST /compute/v2.1/servers => generated 408 bytes in 22363 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:03:23.675239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-345a2ed8-67bf-4ce6-be96-9e8b2da9b826 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:23.675983 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-345a2ed8-67bf-4ce6-be96-9e8b2da9b826 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:23.754136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-345a2ed8-67bf-4ce6-be96-9e8b2da9b826 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:23.774937 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-345a2ed8-67bf-4ce6-be96-9e8b2da9b826 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1245 microversion: 2.1 time: 0.104040 May 23 13:03:23.775796 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 241/487] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:23 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1245 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:24.797104 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:24.799811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:24.811351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:24.812752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:24.814345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:24.816152 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:24.902276 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:24.910841 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fa1c9b4b-1edd-48f5-87d3-4cd74cf1b81f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1249 microversion: 2.1 time: 0.121729 May 23 13:03:24.911562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 247/488] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:24 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1249 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:25.928605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:25.929214 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:25.937764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:25.938236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:25.939001 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:25.939439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:25.955654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=76,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:26.007093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:26.014977 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e974c7da-6b2b-4e0b-abfb-2131377d9957 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1297 microversion: 2.1 time: 0.090591 May 23 13:03:26.015534 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 242/489] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:25 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1297 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:27.033680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:27.034866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:27.041000 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:27.041446 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:27.042272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:27.042708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:27.103960 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:27.113243 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-08f47950-933a-438e-a585-581a7a5839b7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1297 microversion: 2.1 time: 0.085169 May 23 13:03:27.113832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 248/490] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:27 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1297 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:28.132085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:28.132790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:28.140656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:28.141202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:28.142086 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:28.142592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:28.181684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-81148585-1155-425b-8e78-4405cf83aef0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: .version_select at 0x7f9b18106670>, body: {"keypair": {"name": "tempest-keypair-1093406334", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNdXuRwIsXS9LJbrXQ0kNsvb50CO2UrS+KWyEWUBhyLIFO5fSsZ0OeLGZxRXkI8RvVoyEHHAmsVNInjPeXrwEmwdfT5thasUjSc5fi8Vzqp7BVmToAOcG3CUaqd7o5mctw=="}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:28.196011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-81148585-1155-425b-8e78-4405cf83aef0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:03:28.199793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-81148585-1155-425b-8e78-4405cf83aef0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:03:28.226364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:28.234324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ca3ce435-cad7-4821-9244-cdd2852f2575 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1303 microversion: 2.1 time: 0.106419 May 23 13:03:28.242224 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 243/491] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:28 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1303 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:28.286647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-81148585-1155-425b-8e78-4405cf83aef0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.108571 May 23 13:03:28.287410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 249/492] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:03:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:03:29.251540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:29.252150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:29.258800 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:29.259245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:29.260006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:29.260440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:29.393919 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:29.401492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ffeda986-1259-4b1b-a4ac-4f55b919f6fa tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.153877 May 23 13:03:29.402103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 244/493] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:29 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:30.418878 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:30.419587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:30.425537 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:30.426026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:30.426797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:30.427250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:30.562814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:30.570740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a9616194-8090-4ba7-88b3-eea533183d5b tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.155724 May 23 13:03:30.571307 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-426226021", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "27240b79-2fbf-4ada-bcf2-ef920669591c"}], "security_groups": [{"name": "tempest-securitygroup--249631074"}], "key_name": "tempest-keypair-1093406334", "user_data": "}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:30.571833 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 250/494] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:30 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:30.672180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] validate_networks() for [('27240b79-2fbf-4ada-bcf2-ef920669591c', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:03:31.060631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:03:31.061112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:03:31.061840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:03:31.062309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:03:31.062720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:03:31.073912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:31.074652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:31.075997 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:31.076705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:31.095936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:03:31.099992 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:03:31.112916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:03:31.115251 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:03:31.303363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-93292792-935e-4440-b6ea-1e4f1a3171ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.736283 May 23 13:03:31.304018 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 245/495] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:03:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 738 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 23 13:03:31.314496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7f1b4aeb-70ba-4d0c-8f61-f8a972d655ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:31.314962 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7f1b4aeb-70ba-4d0c-8f61-f8a972d655ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:31.370911 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7f1b4aeb-70ba-4d0c-8f61-f8a972d655ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:31.377766 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7f1b4aeb-70ba-4d0c-8f61-f8a972d655ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1241 microversion: 2.1 time: 0.067217 May 23 13:03:31.378396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 251/496] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:31 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1241 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:31.588979 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:31.589599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:31.599108 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:31.599911 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:31.601434 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:31.602367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:31.755561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:31.763409 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-95f80a29-2824-4f2a-8aad-d0e8397e653f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.177848 May 23 13:03:31.763998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 246/497] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:31 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:32.395788 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:32.396185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:32.401930 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:32.402381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:32.403138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:32.404184 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:32.463317 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:32.470638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f6a358d9-b65c-4851-a375-9cb8e39b51b5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1245 microversion: 2.1 time: 0.079502 May 23 13:03:32.471202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 252/498] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:32 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1245 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:32.781263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:32.781807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:32.788256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:32.788706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:32.789467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:32.789933 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:32.917910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:32.925993 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-154285f7-0a2e-4191-9fe2-eae6c7c37692 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.149305 May 23 13:03:32.926558 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 247/499] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:32 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:33.487767 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:33.488998 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:33.495406 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:33.495962 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:33.496841 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:33.498300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:33.498909 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:33.563488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:33.570963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d9c5b29f-e941-4091-b566-4662017f152f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1245 microversion: 2.1 time: 0.087131 May 23 13:03:33.571731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 253/500] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:33 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1245 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:33.943283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:33.944089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:33.950772 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:33.951228 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:33.951984 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:33.952419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:34.182529 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:34.190455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-41fec989-26ec-431c-8fe7-33cfddb52d10 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.251019 May 23 13:03:34.191017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 248/501] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:33 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:34.383075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-8a4f9c4f-6ba2-46c6-91e4-6470892bbc5b req-cdd6f64f-541a-4825-b919-01d0979a3322 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "462787a7-152e-4a6a-9c26-740f68c60a77", "tag": "ca8020dd-47c4-42a6-93d6-b0699126db66"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:34.392168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-8a4f9c4f-6ba2-46c6-91e4-6470892bbc5b req-cdd6f64f-541a-4825-b919-01d0979a3322 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:34.397710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-8a4f9c4f-6ba2-46c6-91e4-6470892bbc5b req-cdd6f64f-541a-4825-b919-01d0979a3322 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:34.406033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-8a4f9c4f-6ba2-46c6-91e4-6470892bbc5b req-cdd6f64f-541a-4825-b919-01d0979a3322 service nova] Creating event network-changed:ca8020dd-47c4-42a6-93d6-b0699126db66 for instance 462787a7-152e-4a6a-9c26-740f68c60a77 on hv-cinder-maste May 23 13:03:34.413263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-8a4f9c4f-6ba2-46c6-91e4-6470892bbc5b req-cdd6f64f-541a-4825-b919-01d0979a3322 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034451 May 23 13:03:34.413969 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 254/502] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:03:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:34.588530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:34.589142 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:34.595702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:34.596147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:34.596947 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:34.597386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:34.689572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:34.703403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-129df0da-316a-40a4-b257-3b1b2da6d8d1 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1293 microversion: 2.1 time: 0.118656 May 23 13:03:34.704341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 249/503] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:34 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1293 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:35.208255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:35.208973 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:35.215266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:35.215716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:35.217210 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:35.217657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:35.542530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:35.553542 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0ade9364-8acf-47fa-bb9c-d58007bf0320 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.348587 May 23 13:03:35.554376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 255/504] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:35 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:35.720988 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:35.723066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:35.731266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:35.732498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:35.733687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:35.734336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:35.820897 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:35.828957 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c88f8bf4-170c-4203-b8f6-8f420f45f40d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1299 microversion: 2.1 time: 0.111677 May 23 13:03:35.829913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 250/505] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:35 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1299 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:36.571545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:36.572241 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:36.578673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:36.579157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:36.579869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:36.580300 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:36.718282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:36.726228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2b700db3-0f12-47ac-a2ff-c61714b8c6c8 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.158644 May 23 13:03:36.726864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 256/506] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:36 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:36.846676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:36.847372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:36.853761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:36.854265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:36.855030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:36.855467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:36.989559 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:36.997115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d1ced0fd-40b7-42a6-a0cf-e22b28bb3fed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.154303 May 23 13:03:36.998261 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 251/507] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:36 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:37.743815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:37.744355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:37.752598 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:37.754331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:37.754522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:37.755197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:37.918221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:37.926311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8d507798-6325-428e-ac7c-2b14d0222d22 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.186383 May 23 13:03:37.926878 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 257/508] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:37 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:38.014269 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:38.014889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:38.021740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:38.022299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:38.023064 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:38.023508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:38.152426 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:38.160584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f082c647-ba52-4359-ab00-37ff8014176b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.150228 May 23 13:03:38.161139 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 252/509] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:38 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:38.943289 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:38.943981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:38.951872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:38.952341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:38.953187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:38.953763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:39.165686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:39.174686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-15c7783a-7247-4e60-a581-5cb769760f59 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.235288 May 23 13:03:39.175406 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 258/510] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:38 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:39.177509 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:39.178158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:39.184329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:39.190016 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:39.190924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:39.191383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:39.330678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:39.337995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5f88a72c-29fb-4f31-97b9-5960f1a72024 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.164327 May 23 13:03:39.338557 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 253/511] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:39 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:39.748313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-608df43c-f0f8-4c75-836a-921e1d51f227 req-fbb0350e-947e-4fe8-bc6c-260f7101e28c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "adac7929-c83b-4f7b-9b97-1b7a48da67a2", "tag": "7f1679f8-1ca4-4c8a-8075-674c69981de0"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:39.756945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-608df43c-f0f8-4c75-836a-921e1d51f227 req-fbb0350e-947e-4fe8-bc6c-260f7101e28c service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:39.757387 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-608df43c-f0f8-4c75-836a-921e1d51f227 req-fbb0350e-947e-4fe8-bc6c-260f7101e28c service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:39.769076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-608df43c-f0f8-4c75-836a-921e1d51f227 req-fbb0350e-947e-4fe8-bc6c-260f7101e28c service nova] Creating event network-changed:7f1679f8-1ca4-4c8a-8075-674c69981de0 for instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 on hv-cinder-maste May 23 13:03:39.774356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-608df43c-f0f8-4c75-836a-921e1d51f227 req-fbb0350e-947e-4fe8-bc6c-260f7101e28c service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030754 May 23 13:03:39.774960 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 259/512] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:03:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:40.191616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:40.192225 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:40.198690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:40.199142 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:40.199944 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:40.200820 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:40.355689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:40.356401 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:40.362763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:40.363446 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:40.364397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:40.364838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:40.406030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:40.413845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-63fab1d5-ab65-4f8a-a4e5-0813ec1b542a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.226156 May 23 13:03:40.414441 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 254/513] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:40 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:40.665777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:40.673380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cd83db9d-0503-4180-be1c-59fb7c98da54 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.322095 May 23 13:03:40.673986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 260/514] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:40 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:41.430626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:41.431250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:41.433073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:41.433726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=8 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:41.439774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:41.440231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:41.440999 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:41.441445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:41.571196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:41.578923 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-996b37b3-f8b3-4b7c-a6ef-b0042db2f216 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.152207 May 23 13:03:41.579489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 255/515] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:41 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:41.691331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:41.692124 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:41.695613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=12 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:41.699393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:41.699844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:41.704258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:41.704708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:41.833261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:41.840676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-13f4bb7f-ed8c-4abd-a850-27ab76d9286f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.153193 May 23 13:03:41.841236 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 261/516] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:41 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:42.596476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:42.597210 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:42.606324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:42.608373 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:42.609276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:42.610121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:42.748436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:42.755834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2a2b9740-bd90-4e35-817e-fad4e36b6291 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.163666 May 23 13:03:42.756395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 256/517] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:42 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:42.858803 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:42.859333 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:42.865279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:42.865728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:42.866528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:42.866964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:42.995627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:43.003705 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-18bc0790-35ea-42a9-b13b-69cd664cf678 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.149745 May 23 13:03:43.004264 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 262/518] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:42 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:43.773469 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:43.774107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:43.786324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:43.786765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:43.787552 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:43.787981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:43.933379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:43.941637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d8b6672a-a319-4b32-b576-b6ec2ba80489 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.172716 May 23 13:03:43.942338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 257/519] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:43 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:44.020747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:44.021368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:44.027364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:44.027810 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:44.028715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:44.029177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:44.155507 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:44.162950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6080cb8f-a82e-4fe2-bfcc-551d96634bd3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.146324 May 23 13:03:44.163456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 263/520] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:44 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:44.959232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:44.959843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:44.966456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:44.966969 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:44.967737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:44.968175 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:45.099788 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:45.107905 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3b365b86-5918-441d-a647-26ac50784420 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.152136 May 23 13:03:45.108833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 258/521] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:44 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:45.180215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:45.180934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:45.187070 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:45.187459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:45.188235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:45.188720 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:45.331924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:45.339328 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a6fe77bf-e28b-43f4-88f3-d70c12d12929 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.162982 May 23 13:03:45.339879 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 264/522] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:45 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:46.124921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:46.125563 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:46.132023 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:46.132476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:46.133239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:46.133674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:46.148233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:46.261124 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:46.268769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3a596148-1043-4ff4-b3aa-4d92e9536dfc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.147488 May 23 13:03:46.269329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 259/523] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:46 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:46.356388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:46.357021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:46.363567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:46.364007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:46.364752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:46.365178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:46.495859 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:46.503708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a6174ae9-3c48-4b0a-a9ba-6d850367ec86 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.150969 May 23 13:03:46.504192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 265/524] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:46 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:47.285357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:47.285914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:47.292314 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:47.292755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:47.293504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:47.293961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:47.421098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:47.428331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-90f7a5a6-3a98-4aaa-b57e-c129dce34cf4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.146945 May 23 13:03:47.428887 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 260/525] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:47 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:47.521577 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:47.522221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:47.528119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:47.528562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:47.529309 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:47.529735 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:47.547908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:47.659051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:47.666378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-31f6a9e6-33c5-44e7-b7fa-3508d1fc2b41 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.148911 May 23 13:03:47.666931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 266/526] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:47 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:48.444353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:48.444970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:48.452044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:48.452493 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:48.453255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:48.453698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:48.582708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:48.590094 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9e284f57-a373-4c7d-9922-1b79c7f25cf9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.149562 May 23 13:03:48.590656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 261/527] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:48 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:48.683090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:48.683678 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:48.690585 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:48.691029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:48.691785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:48.692236 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:48.822145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:48.831153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-33bf27ec-5a2b-41cb-81e8-3f69eeada026 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.151930 May 23 13:03:48.836734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 267/528] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:48 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:49.606617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:49.607219 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:49.609500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:49.614322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:49.614765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:49.615521 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:49.615955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:49.755866 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:49.764546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-99e202a8-278e-4dbd-b21f-690e706ed8f7 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.161267 May 23 13:03:49.765274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 262/529] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:49 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:49.848282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:49.849528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:49.855622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:49.856079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:49.856854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:49.857717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:50.090177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:50.097706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d8c2c2fe-e095-4eca-8a03-19bf16238440", "device": "/dev/sdb"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:50.099480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:50.101011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b3bd7cee-0a8b-436e-9af6-9498fa366fa2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.256413 May 23 13:03:50.101570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 268/530] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:49 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:50.105531 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:50.106051 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:50.107036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:50.107652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:50.782266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:50.782861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:50.788932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:50.789249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:50.790088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:50.790522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:50.932987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:50.941510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-54ab606a-f442-4cfa-a652-16e87e23a1f9 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.163345 May 23 13:03:50.942121 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 269/531] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:50 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:50.959109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2639f2fd-d568-4676-866e-a2592c408679 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "POST /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.864627 May 23 13:03:50.959609 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 263/532] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:03:50 2022] POST /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617/os-volume_attachments => generated 194 bytes in 867 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:51.121081 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:51.121642 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:51.128321 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:51.133433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:51.133433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:51.133433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:51.259559 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:51.267646 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-27ca12f8-af2d-41b5-880d-b898ef0a1520 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.150680 May 23 13:03:51.268214 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 270/533] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:51 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:51.959798 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:51.960490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:51.966697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:51.967800 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:51.968637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:51.969078 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:52.105452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:52.113469 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fb7876c7-9515-4529-900f-58eb3da11a42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.158281 May 23 13:03:52.114070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 264/534] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:51 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:52.284816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:52.285346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:52.288319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=4 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:03:52.291745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:52.292193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:52.292997 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:52.293434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:52.424132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:52.429830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-42f6f741-9345-4c32-ae79-af08099446f3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.149047 May 23 13:03:52.430418 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 271/535] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:52 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:53.130655 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:53.131268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:53.137650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:53.138152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:53.139008 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:53.139960 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:53.215215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-10a28788-449d-4e84-896b-342e638805d6 req-24c7a2be-221b-4dd2-9a47-f769589c78d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "462787a7-152e-4a6a-9c26-740f68c60a77", "name": "network-vif-plugged", "status": "completed", "tag": "ca8020dd-47c4-42a6-93d6-b0699126db66"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:53.226069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-10a28788-449d-4e84-896b-342e638805d6 req-24c7a2be-221b-4dd2-9a47-f769589c78d3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:53.226522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-10a28788-449d-4e84-896b-342e638805d6 req-24c7a2be-221b-4dd2-9a47-f769589c78d3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:53.238181 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-10a28788-449d-4e84-896b-342e638805d6 req-24c7a2be-221b-4dd2-9a47-f769589c78d3 service nova] Creating event network-vif-plugged:ca8020dd-47c4-42a6-93d6-b0699126db66 for instance 462787a7-152e-4a6a-9c26-740f68c60a77 on hv-cinder-maste May 23 13:03:53.243720 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-10a28788-449d-4e84-896b-342e638805d6 req-24c7a2be-221b-4dd2-9a47-f769589c78d3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032132 May 23 13:03:53.244965 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 272/536] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:03:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:53.297408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:53.305125 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c8cb1f53-12d3-49e4-9cde-6703624d1319 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.178288 May 23 13:03:53.305682 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 265/537] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:53 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:53.447236 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:53.447964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:53.454292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:53.454771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:53.455531 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:53.455969 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:53.587878 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:53.595993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0ba0eeab-6fa1-48a3-abf8-ee040e5c62ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.152926 May 23 13:03:53.596722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 273/538] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:53 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:54.331311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:54.331935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:54.339038 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:54.339489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:54.340274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:54.340785 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:54.613419 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:54.614062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:54.622383 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:54.622833 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:54.626238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:54.626883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:54.677127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:54.684680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3793b612-83ee-40b7-ac3b-75e31c333513 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.358152 May 23 13:03:54.685257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 266/539] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:54 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:54.761971 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:54.770226 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1699d8a7-9266-4b7d-ace3-2518bae3ce65 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.160652 May 23 13:03:54.770798 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 274/540] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:54 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:55.702115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:55.702983 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:55.711242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:55.711863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:55.712746 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:55.713419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:55.804533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:55.805239 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:55.814935 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:55.815836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:55.817552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:55.825639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:55.904620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:55.913841 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e52eff2a-c553-43d2-88ad-c3436ee84cb4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.216137 May 23 13:03:55.914581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 267/541] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:55 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:56.049750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:56.064495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d99d7bb1-0579-4585-a9f8-dbf84efaf81d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.274240 May 23 13:03:56.065621 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 275/542] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:55 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:56.931286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:56.931891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:56.939039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:56.939486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:56.940251 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:56.940689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:57.071671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:57.080574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cf4739b9-f1b8-494c-966b-908ffe990c7f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1370 microversion: 2.1 time: 0.153287 May 23 13:03:57.081441 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 268/543] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:56 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:57.081631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:57.082372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:57.088251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:57.088694 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:57.089445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:57.089872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:57.218622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:57.225845 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3e212b91-6482-4ff6-8b5a-f918919d4921 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.148035 May 23 13:03:57.226432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 276/544] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:57 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:58.098258 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:58.098990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:58.105354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.105711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-95825282-4ae7-49f3-8ba4-9703c90a0cfb req-1eeca6a6-6881-41ef-b372-507880a92c11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "adac7929-c83b-4f7b-9b97-1b7a48da67a2", "name": "network-vif-plugged", "status": "completed", "tag": "7f1679f8-1ca4-4c8a-8075-674c69981de0"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:58.106199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.106615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.107042 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.115823 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-95825282-4ae7-49f3-8ba4-9703c90a0cfb req-1eeca6a6-6881-41ef-b372-507880a92c11 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.116269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-95825282-4ae7-49f3-8ba4-9703c90a0cfb req-1eeca6a6-6881-41ef-b372-507880a92c11 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.128890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-95825282-4ae7-49f3-8ba4-9703c90a0cfb req-1eeca6a6-6881-41ef-b372-507880a92c11 service nova] Creating event network-vif-plugged:7f1679f8-1ca4-4c8a-8075-674c69981de0 for instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 on hv-cinder-maste May 23 13:03:58.133828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-95825282-4ae7-49f3-8ba4-9703c90a0cfb req-1eeca6a6-6881-41ef-b372-507880a92c11 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031651 May 23 13:03:58.134453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 277/545] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:03:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:58.243829 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:58.244759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:58.245851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:58.252766 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.253297 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.253730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2d0976a9-4d07-4568-8596-687dc2fb03fe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1565 microversion: 2.1 time: 0.159923 May 23 13:03:58.254269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.254405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 269/546] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:58 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1565 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:03:58.255045 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.270706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:58.275043 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:58.277031 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.277479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.278329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:58.278757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:58.384760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c064ac3f-b242-4a48-949f-47729083e32c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.118491 May 23 13:03:58.385556 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 270/547] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:03:58 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-interface => generated 285 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:03:58.435819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:58.443927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-78af9dc1-de62-4e77-803a-183eded5af5c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.204495 May 23 13:03:58.444491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 278/548] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:58 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:03:59.462112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:03:59.463451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:03:59.474900 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:59.475733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:59.477013 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:59.477646 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:59.603723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-c46fca95-1e2e-48b5-8541-f038924f795e req-50816aa4-c5e3-4c58-b3fa-4c15a687012b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "462787a7-152e-4a6a-9c26-740f68c60a77", "tag": "ca8020dd-47c4-42a6-93d6-b0699126db66"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:03:59.617034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-c46fca95-1e2e-48b5-8541-f038924f795e req-50816aa4-c5e3-4c58-b3fa-4c15a687012b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:03:59.617476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-c46fca95-1e2e-48b5-8541-f038924f795e req-50816aa4-c5e3-4c58-b3fa-4c15a687012b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:03:59.634739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-c46fca95-1e2e-48b5-8541-f038924f795e req-50816aa4-c5e3-4c58-b3fa-4c15a687012b service nova] Creating event network-changed:ca8020dd-47c4-42a6-93d6-b0699126db66 for instance 462787a7-152e-4a6a-9c26-740f68c60a77 on hv-cinder-maste May 23 13:03:59.639573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-c46fca95-1e2e-48b5-8541-f038924f795e req-50816aa4-c5e3-4c58-b3fa-4c15a687012b service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039933 May 23 13:03:59.640171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 279/549] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:03:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:03:59.663114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:03:59.670378 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3e2a0d5c-7ae3-48b1-91d4-90438484f510 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.212748 May 23 13:03:59.670939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 271/550] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:03:59 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:00.687661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:00.688354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:00.694670 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:00.695111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:00.695868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:00.696751 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:00.824750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:00.832864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-adfc422b-8ed6-4bf8-9f9f-5e2287ded57f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.149387 May 23 13:04:00.834614 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 280/551] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:00 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:04:01.849893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:01.850492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:01.856862 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:01.857555 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:01.858185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:01.858563 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:02.025100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:02.032664 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4924cc88-9108-46fa-8c90-2188b50c1bbb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.186391 May 23 13:04:02.033122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 272/552] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:01 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:03.048890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:03.049500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:03.055459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:03.055904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:03.056660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:03.057096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:03.191430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:03.198792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-edb6bb49-1759-4e46-99c0-ff7e821ddf87 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.154206 May 23 13:04:03.199295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 281/553] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:03 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:04.215203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:04.215817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:04.222673 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:04.223125 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:04.223894 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:04.224335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:04.364877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:04.372283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-594ef404-ca49-40a9-b7e3-55c1b70b95e3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.160961 May 23 13:04:04.372836 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 273/554] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:04 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:05.389786 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:05.390442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:05.396745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:05.397194 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:05.397975 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:05.398419 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:05.525523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:05.533694 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fc26a8a0-e0c5-4164-8936-ed8ae5a2ebcd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.147506 May 23 13:04:05.534622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 282/555] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:05 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:06.551162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:06.551776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:06.558213 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:06.558662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:06.559421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:06.559856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:06.695517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:06.702830 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cfd28d45-8ae8-4255-9437-2ecf93301dba tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.156078 May 23 13:04:06.703386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 274/556] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:06 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:07.721453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:07.722251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:07.728681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:07.729125 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:07.729897 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:07.730327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:07.856978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:07.864863 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-499275f6-80c9-432b-a363-4daf67f81d3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.148117 May 23 13:04:07.865414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 283/557] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:07 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:08.882199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:08.882826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:08.889312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:08.889757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:08.890549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:08.890986 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:08.894499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:09.024806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:09.031913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7c4d3c19-0fa8-48ee-b3b1-796f582dedd9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1366 microversion: 2.1 time: 0.153624 May 23 13:04:09.032717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 275/558] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:08 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:10.048760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:10.049358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:10.055374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:10.055819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:10.056573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:10.057005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:10.184882 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:10.193513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a7f9b4d2-9cc3-47ce-bef6-d42ee054360e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1556 microversion: 2.1 time: 0.148649 May 23 13:04:10.194114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 284/559] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:10 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1556 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:10.211221 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:10.211767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:10.217711 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:10.218190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:10.218943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:10.219371 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:10.296205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e7828cbf-c5dc-47a1-bbc9-23038c3dc202 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.089766 May 23 13:04:10.296797 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 276/560] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:04:10 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/os-interface => generated 285 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:04:11.297529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-4ce7b9a0-f63f-4714-b836-0bcd8da85e77 req-e6bbd34f-29bb-451f-baec-afd508a0b013 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "adac7929-c83b-4f7b-9b97-1b7a48da67a2", "tag": "7f1679f8-1ca4-4c8a-8075-674c69981de0"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:04:11.306583 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-4ce7b9a0-f63f-4714-b836-0bcd8da85e77 req-e6bbd34f-29bb-451f-baec-afd508a0b013 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:11.307093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-4ce7b9a0-f63f-4714-b836-0bcd8da85e77 req-e6bbd34f-29bb-451f-baec-afd508a0b013 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:11.312529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:11.321719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-4ce7b9a0-f63f-4714-b836-0bcd8da85e77 req-e6bbd34f-29bb-451f-baec-afd508a0b013 service nova] Creating event network-changed:7f1679f8-1ca4-4c8a-8075-674c69981de0 for instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 on hv-cinder-maste May 23 13:04:11.327017 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-4ce7b9a0-f63f-4714-b836-0bcd8da85e77 req-e6bbd34f-29bb-451f-baec-afd508a0b013 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033098 May 23 13:04:11.327953 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 285/561] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:04:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:04:29.180139 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=29 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:29.180923 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=9 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:29.185004 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:29.185499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:29.192297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:29.192863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:29.193572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:29.194041 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:29.319776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lazy-loading 'services' on Instance uuid 4cae571e-2b00-4426-a0e8-aa2c62e48617 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:04:29.568586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-09e19e20-8881-4b14-9f79-311b92d6d6da tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "DELETE /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617/os-volume_attachments/d8c2c2fe-e095-4eca-8a03-19bf16238440" status: 202 len: 0 microversion: 2.1 time: 0.389440 May 23 13:04:29.569087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 277/562] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:04:29 2022] DELETE /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617/os-volume_attachments/d8c2c2fe-e095-4eca-8a03-19bf16238440 => generated 0 bytes in 391 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:04:31.741559 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:31.744658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:31.744658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=29 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:31.749413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:31.749863 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:31.750664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:31.751132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:31.775729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:04:32.043062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-275da9c4-69d2-4eb2-98ae-ff913b15e3a1 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "DELETE /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 204 len: 0 microversion: 2.1 time: 0.305244 May 23 13:04:32.043578 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 286/563] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:04:31 2022] DELETE /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:04:32.053318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:32.054077 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:32.060303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:32.063382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:32.063382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:32.063382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:32.192869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:32.200361 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5abb45e0-80ab-43fc-8349-dbbd567c399d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1513 microversion: 2.1 time: 0.152376 May 23 13:04:32.200919 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 278/564] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:32 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1513 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:33.219070 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:33.219754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:33.226462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:33.226906 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:33.227662 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:33.228090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:33.398159 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:33.402600 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f21eabaa-6e5b-4480-a49e-4c58406703b7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1513 microversion: 2.1 time: 0.187968 May 23 13:04:33.403178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 287/565] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:33 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1513 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:34.420833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:34.421530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:34.428138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:34.428603 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:34.429421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:34.429902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:34.643478 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:34.651134 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2a01a3a8-28f2-4023-ad55-d4d5351f71ed tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1513 microversion: 2.1 time: 0.234240 May 23 13:04:34.651709 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 279/566] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:34 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1513 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:35.028101 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-19895526-7495-4b99-b38e-9b6e512ab18b req-2ffd4e87-b61c-45b4-8ee7-49709077aa0e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4cae571e-2b00-4426-a0e8-aa2c62e48617", "name": "network-vif-deleted", "tag": "6e114242-9ed0-40d0-98db-97f57599dd87"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:04:35.036485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-19895526-7495-4b99-b38e-9b6e512ab18b req-2ffd4e87-b61c-45b4-8ee7-49709077aa0e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:35.036931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-19895526-7495-4b99-b38e-9b6e512ab18b req-2ffd4e87-b61c-45b4-8ee7-49709077aa0e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:35.047685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-19895526-7495-4b99-b38e-9b6e512ab18b req-2ffd4e87-b61c-45b4-8ee7-49709077aa0e service nova] Creating event network-vif-deleted:6e114242-9ed0-40d0-98db-97f57599dd87 for instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 on hv-cinder-maste May 23 13:04:35.053340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-19895526-7495-4b99-b38e-9b6e512ab18b req-2ffd4e87-b61c-45b4-8ee7-49709077aa0e service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028934 May 23 13:04:35.053943 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 288/567] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:04:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:04:35.668972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:35.669584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:35.676835 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:35.677284 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:35.678089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:35.678534 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:35.708893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:35.709376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:35.709849 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lazy-loading 'fault' on Instance uuid 4cae571e-2b00-4426-a0e8-aa2c62e48617 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:04:35.750418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:35.757972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0b12d013-f908-4211-bc4c-c04a192f2be2 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 200 len: 1308 microversion: 2.1 time: 0.092761 May 23 13:04:35.758541 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 280/568] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:35 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 1308 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:36.775795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:36.778065 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 4cae571e-2b00-4426-a0e8-aa2c62e48617] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:36.782813 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:36.783257 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:36.784105 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:36.784528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:36.797396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] HTTP exception thrown: Instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 could not be found. May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Request method failure captured: May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 HTTP/1.1 May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4XXq-uWfwZEWkYRO-aadJr-HTlQ9NZcOW6Bov--ZXryPok-tdln_HuTrEB7UR2xv4UQ1qA1H9KBdvAWr1u9A3z0zvgPxPnN9xWr9hIOZD-6T_umRMjW12Z7eCp-f-Z4e1lg4HjRoMfcw1Vy_eAxJrT1Vp6JrL2UCjORc-kpwKctJTA May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: ccd27933078b44b48323995e98b21318 May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesSnapshotTestJSON-154686593 May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:04:36.799543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_ccd27933078b44b48323995e98b21318"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/ccd27933078b44b48323995e98b21318"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/ccd27933078b44b48323995e98b21318"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/ccd27933078b44b48323995e98b21318"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesSnapshotTestJSON-154686593 May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: ccd27933078b44b48323995e98b21318 May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-154686593 May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesSnapshotTestJSON-154686593-project May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 5c6147fed25a4212b6175aacc460943d May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesSnapshotTestJSON-154686593-project May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 could not be found. May 23 13:04:36.801256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '4cae571e-2b00-4426-a0e8-aa2c62e48617'} May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 could not be found. May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:04:36.802254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 could not be found. May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 could not be found. May 23 13:04:36.803753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:04:36.805008 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Returning 404 to user: Instance 4cae571e-2b00-4426-a0e8-aa2c62e48617 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:04:36.805008 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cc491e03-1905-4367-b064-628bb9709f26 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617" status: 404 len: 111 microversion: 2.1 time: 0.029368 May 23 13:04:36.805008 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 289/569] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:36 2022] GET /compute/v2.1/servers/4cae571e-2b00-4426-a0e8-aa2c62e48617 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:04:50.198985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2094287757", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "networks": [{"uuid": "f040d182-eca3-4e6d-946e-e964e751bb04"}]}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:04:50.206876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=10 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:50.207527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=29,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:50.251453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] validate_networks() for [('f040d182-eca3-4e6d-946e-e964e751bb04', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:04:50.812872 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:04:50.813292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:04:50.813795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:04:50.814237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:04:50.814657 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:04:50.823410 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:50.827593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:50.827593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:50.827593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:50.841587 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Getting quotas for project ccd27933078b44b48323995e98b21318. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:04:50.845591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Getting quotas for user 5c6147fed25a4212b6175aacc460943d and project ccd27933078b44b48323995e98b21318. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:04:50.856939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:04:50.859204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:04:50.969958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7ac53bfa-6f96-4f90-b3f6-6bc86268a11f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.774691 May 23 13:04:50.970520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 281/570] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:04:50 2022] POST /compute/v2.1/servers => generated 380 bytes in 776 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:04:50.980858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6c96584f-2470-4bae-9426-09f29253a019 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:50.981462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6c96584f-2470-4bae-9426-09f29253a019 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:50.983697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:50.984441 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:04:51.039138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6c96584f-2470-4bae-9426-09f29253a019 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:51.047081 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6c96584f-2470-4bae-9426-09f29253a019 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1219 microversion: 2.1 time: 0.069942 May 23 13:04:51.047870 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 290/571] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:50 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1219 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:52.065669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:52.066455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:52.073311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:52.073760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:52.077352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:52.077352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:52.138439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:52.146613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-24faf20b-3129-446e-a249-e283e1a8b316 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1223 microversion: 2.1 time: 0.086236 May 23 13:04:52.148572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 282/572] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:52 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1223 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:53.164216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:53.165719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:53.172437 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:53.172880 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:53.173630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:53.174186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:53.264937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:53.273682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cdadb89a-1b9b-4dea-91b2-f9cdb1d95f83 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1279 microversion: 2.1 time: 0.113651 May 23 13:04:53.275929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 291/573] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:53 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1279 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:54.292745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:54.293356 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:54.299839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:54.300288 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:54.303926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:54.303926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:54.490481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:54.498158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e591cd2c-5861-4abe-a8a0-4e6cca197dc5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.209446 May 23 13:04:54.498730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 283/574] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:54 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:54.759537 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-28c0c792-83d1-4c0a-bc7a-5cce742ed5fd req-b921d03a-49bc-487c-a6ef-0822c75822ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f", "tag": "e027552b-2e69-458a-acf3-a19901e63f06"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:04:54.768402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-28c0c792-83d1-4c0a-bc7a-5cce742ed5fd req-b921d03a-49bc-487c-a6ef-0822c75822ff service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:54.769106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-28c0c792-83d1-4c0a-bc7a-5cce742ed5fd req-b921d03a-49bc-487c-a6ef-0822c75822ff service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:54.781489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-28c0c792-83d1-4c0a-bc7a-5cce742ed5fd req-b921d03a-49bc-487c-a6ef-0822c75822ff service nova] Creating event network-changed:e027552b-2e69-458a-acf3-a19901e63f06 for instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f on hv-cinder-maste May 23 13:04:54.786697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-28c0c792-83d1-4c0a-bc7a-5cce742ed5fd req-b921d03a-49bc-487c-a6ef-0822c75822ff service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031754 May 23 13:04:54.787295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 292/575] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:04:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:04:55.518671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:55.519486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:55.527107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:55.527429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:55.528188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:55.528615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:55.666304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:55.674104 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-59bb4ea2-a356-4095-aec8-4356bdf85e5f tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.159204 May 23 13:04:55.677439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 284/576] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:55 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:56.693708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:56.694422 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:56.701045 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:56.701497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:56.702306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:56.702743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:56.840764 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:56.848022 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dcb22841-e252-4f5c-ad32-8b15e5e4f4ce tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.159346 May 23 13:04:56.848584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 293/577] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:56 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:57.866369 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:57.867164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:57.874028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:57.874479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:57.875306 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:57.875738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:58.225205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:58.232500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5c6cad39-28b4-49b4-9bad-880d931d0b19 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.370355 May 23 13:04:58.235530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 285/578] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:57 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:04:59.249870 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:04:59.250603 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:04:59.257112 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:59.257553 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:59.258356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:04:59.258787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:04:59.546013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:04:59.554275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9fdfe862-fe4a-407d-87a4-2e5c77189dab tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.308379 May 23 13:04:59.555004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 294/579] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:04:59 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:00.572602 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:00.573140 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:00.579599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:00.580051 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:00.580832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:00.581266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:00.716117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:00.723650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-66ee2ebc-4b72-4888-ab9e-1ad881c3fa80 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.156312 May 23 13:05:00.724221 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 286/580] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:00 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:01.740773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:01.741384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:01.743412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:01.748226 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:01.753010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:01.753010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:01.753010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:01.879286 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:01.887776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d25a7cd5-fe5a-419e-b16b-c79f20838513 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.151122 May 23 13:05:01.888337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 295/581] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:01 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:02.471145 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-bb1ebb70-48ef-4e6c-9656-abe7e09d9b40 req-aa2ad4a7-8f5c-4c2b-a27e-d5fc97e834f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f", "name": "network-vif-plugged", "status": "completed", "tag": "e027552b-2e69-458a-acf3-a19901e63f06"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:05:02.474656 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:02.475364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:02.481352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-bb1ebb70-48ef-4e6c-9656-abe7e09d9b40 req-aa2ad4a7-8f5c-4c2b-a27e-d5fc97e834f9 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:02.481802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-bb1ebb70-48ef-4e6c-9656-abe7e09d9b40 req-aa2ad4a7-8f5c-4c2b-a27e-d5fc97e834f9 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:02.492914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-bb1ebb70-48ef-4e6c-9656-abe7e09d9b40 req-aa2ad4a7-8f5c-4c2b-a27e-d5fc97e834f9 service nova] Creating event network-vif-plugged:e027552b-2e69-458a-acf3-a19901e63f06 for instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f on hv-cinder-maste May 23 13:05:02.497760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-bb1ebb70-48ef-4e6c-9656-abe7e09d9b40 req-aa2ad4a7-8f5c-4c2b-a27e-d5fc97e834f9 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030727 May 23 13:05:02.498374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 287/582] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:05:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:05:02.910114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:02.910811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:02.916797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:02.917237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:02.918071 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:02.918491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:03.045096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:03.053044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-95e9b25e-f309-4b43-b585-30777b71b299 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.146303 May 23 13:05:03.054143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 296/583] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:02 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:05:04.071446 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:04.072063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:04.079223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:04.079673 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:04.080423 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:04.080864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:04.208002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:04.215826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fb000dcc-963b-445f-8fde-7f17ff530445 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.149871 May 23 13:05:04.216392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 288/584] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:04 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:04.807701 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:05:04.807916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:05:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:05:05.232229 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:05.232839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:05.238830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:05.239279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:05.240765 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:05.241215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:05.372001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:05.379392 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8e658390-b485-4bce-a36d-50e2f267dff7 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1319 microversion: 2.1 time: 0.151022 May 23 13:05:05.379944 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 297/585] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:05 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:06.397242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:06.397912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:06.404433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:06.404882 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:06.405642 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:06.406111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:06.540425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:06.542004 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a1b45927-0923-4a37-beba-b2bb33de19ba tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1507 microversion: 2.1 time: 0.149986 May 23 13:05:06.542556 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 289/586] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:06 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1507 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:06.559134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "878dfd6f-b6f4-43cb-bed7-8e1d114aebe2", "device": "/dev/sdb"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:05:06.560685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:06.566650 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:06.578615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:06.578615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:06.578615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:07.275196 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5a95f0a1-a923-4984-8635-2cf492013ae6 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "POST /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.719702 May 23 13:05:07.276568 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 298/587] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:05:06 2022] POST /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f/os-volume_attachments => generated 194 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:05:12.827920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:12.828483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:12.835148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:12.835591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:12.836343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:12.836774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:12.859838 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-26f116e0-0188-4eb0-a066-0238d981a32d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/ips" status: 200 len: 152 microversion: 2.20 time: 0.035897 May 23 13:05:12.860395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 290/588] 10.222.0.30 () {60 vars in 1288 bytes} [Mon May 23 13:05:12 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/ips => generated 152 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:05:16.096654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:05:16.102563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:16.112153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:16.112495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:16.113281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:16.113717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:16.138117 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lazy-loading 'flavor' on Instance uuid d16a2136-f506-4eb4-8f03-5dadb2a6b515 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:05:16.700297 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-37b7dbe6-a6a4-41ac-8cc7-101ccd4eb645 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/action" status: 202 len: 0 microversion: 2.20 time: 0.609210 May 23 13:05:16.701055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 299/589] 10.222.0.30 () {62 vars in 1318 bytes} [Mon May 23 13:05:16 2022] POST /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/action => generated 0 bytes in 611 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 13:05:16.710489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:16.711112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:16.718072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:16.718522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:16.719298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:16.719732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:16.862338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:16.870334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:16.871812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:16.873013 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-59728c71-a3eb-47ee-b6cf-f9772e189b86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.167781 May 23 13:05:16.873557 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 291/590] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:16 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:17.893849 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:17.894560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:17.901776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:17.902243 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:17.902988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:17.903419 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:18.032207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:18.040827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:18.042336 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:18.043471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e10a6504-930a-4ff3-a36a-a63841a7c026 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.154484 May 23 13:05:18.044041 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 300/591] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:17 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:19.061586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:19.062220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:19.069316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:19.069758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:19.070534 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:19.070961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:19.198074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:19.205925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:19.207414 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:19.209493 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f7947f72-1e77-4f39-ba62-93d3aa65751c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.150796 May 23 13:05:19.210264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 292/592] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:19 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:20.227592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:20.228191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:20.234649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:20.235087 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:20.235833 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:20.236256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:20.364219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:20.372219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:20.373668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:20.374825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9a34c6bd-8da4-4486-89cd-55a5d0d4120b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.151150 May 23 13:05:20.375375 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 301/593] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:20 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:21.392636 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:21.393249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:21.395916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:21.401123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:21.401598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:21.402370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:21.402808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:21.405384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=63 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:21.533862 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:21.542511 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:21.544007 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:21.545146 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-717623b9-9b89-4ddb-b7fd-94ca21cc8b96 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.156389 May 23 13:05:21.545704 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 293/594] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:21 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:22.564100 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:22.564755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:22.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:22.577136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:22.577136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:22.577136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:22.712504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:22.720496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:22.722026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:22.723172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-63119156-e93e-4e8f-8ff9-4f6cb9a6754e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.163176 May 23 13:05:22.723720 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 302/595] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:22 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:23.741718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:23.742344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:23.748896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:23.749441 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:23.750264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:23.750696 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:23.877682 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:23.885584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:23.887103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:23.888238 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-603d61ee-9f8f-4bd8-a773-d690321d5b11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.150414 May 23 13:05:23.888783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 294/596] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:23 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:24.906362 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:24.906963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:24.909284 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=29 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:24.914255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:24.920453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:24.920453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:24.920453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:24.920453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:25.049072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:25.057344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:25.058855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:25.059994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e0579930-8297-479f-b756-d2726eecbca0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.157558 May 23 13:05:25.060546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 303/597] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:24 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:26.078421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:26.078954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:26.085380 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:26.085825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:26.087203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:26.087641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:26.217432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:26.225184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:26.227254 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:26.228384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-61c98dce-0195-40f3-a480-b8ac2c9cf34c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.153943 May 23 13:05:26.228931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 295/598] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:26 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:27.246563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:27.247165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:27.253646 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:27.254132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:27.254878 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:27.255307 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:27.381196 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:27.389743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:27.391270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:27.392417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-917fbb9a-1673-4ee0-913f-6a7a8fb5315b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.149759 May 23 13:05:27.392968 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 304/599] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:27 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:28.410955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:28.411594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:28.418947 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:28.419397 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:28.420147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:28.420654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:28.555462 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:28.564557 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:28.566303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:28.567749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-507a8c3d-7539-48a0-8dc6-be0108125a0a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.160808 May 23 13:05:28.568565 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 296/600] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:28 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:29.589354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:29.591416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:29.598063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:29.598516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:29.599312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:29.599756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:29.728649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:29.736601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:29.738075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:29.739223 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-128cebcf-df7e-460d-b2ed-30e4251feb88 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.154592 May 23 13:05:29.739779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 305/601] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:29 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:30.757870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:30.759220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:30.765217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:30.765732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:30.766543 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:30.766982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:30.897799 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:30.907002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:30.908491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:30.909635 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c8a40193-caa7-4c3b-bbd8-dcbb130a58a5 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.155965 May 23 13:05:30.910223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 297/602] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:30 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:31.339140 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:31.339655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:31.346046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:31.346499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:31.347261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:31.347693 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:31.476434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lazy-loading 'services' on Instance uuid 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:05:31.765982 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-073be74d-5bfc-4626-bd2d-cc65e3b298e5 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "DELETE /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f/os-volume_attachments/878dfd6f-b6f4-43cb-bed7-8e1d114aebe2" status: 202 len: 0 microversion: 2.1 time: 0.431543 May 23 13:05:31.766602 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 306/603] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:05:31 2022] DELETE /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f/os-volume_attachments/878dfd6f-b6f4-43cb-bed7-8e1d114aebe2 => generated 0 bytes in 433 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:05:31.928420 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:31.929027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:31.936568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:31.937138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:31.937929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:31.938379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:32.067338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:32.075877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:32.077357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:32.078522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d9f9e6fe-940f-4811-93f4-6878bc4ff052 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.153990 May 23 13:05:32.079100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 298/604] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:31 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:33.099487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:33.100235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:33.112612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:33.113059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:33.113835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:33.114350 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:33.268730 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:33.277334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:33.278902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:33.282996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bb29d7d8-a872-46af-a6df-af85a62f74db tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.187028 May 23 13:05:33.282996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 307/605] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:33 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:33.939451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:33.939971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:33.946919 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:33.947316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:33.948069 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:33.948497 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:33.972732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:05:34.245858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b03200c6-5993-4fb2-9ee7-b24b32c0a8de tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "DELETE /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 204 len: 0 microversion: 2.1 time: 0.309862 May 23 13:05:34.246405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 299/606] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:05:33 2022] DELETE /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:05:34.254647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:34.258819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:34.262941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:34.263392 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:34.264149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:34.264581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:34.299543 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:34.300235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:34.307085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:34.307537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:34.308307 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:34.308746 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:34.444505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:34.452146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c94ed6f9-559d-4e70-8ba5-41c678055c3e tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1513 microversion: 2.1 time: 0.201450 May 23 13:05:34.452711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 308/607] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:34 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1513 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:34.473794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:34.482540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:34.484027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:34.485168 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b2297728-a895-4f79-9d94-7ba471bc9da6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.189940 May 23 13:05:34.485714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 300/608] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:34 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:05:35.469843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:35.470480 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:35.476948 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:35.477915 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:35.478685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:35.479122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:35.503233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:35.503839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:35.510386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:35.510833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:35.511582 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:35.512021 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:35.666497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:35.673844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d35e001b-3e96-430a-bf4d-ccc84af01d5d tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1513 microversion: 2.1 time: 0.207831 May 23 13:05:35.674529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 309/609] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:35 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1513 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:35.688076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:35.696087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:35.697576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:35.698741 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-55912998-7cdb-41e1-9889-9887f86d35f1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.199349 May 23 13:05:35.699288 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 301/610] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:35 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:05:36.691494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:36.692094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:36.699123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:36.699564 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:36.700320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:36.700747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:36.717260 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:36.717787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:36.724824 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:36.725526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:36.726327 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:36.727728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:36.959506 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:36.966422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:36.967131 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-df068047-89c7-417c-98a7-776d15372e28 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1513 microversion: 2.1 time: 0.279870 May 23 13:05:36.967749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 310/611] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:36 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1513 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:36.974543 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:36.976019 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:36.977774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e03c0a28-7775-49c4-8986-b4e1b91360be tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.263849 May 23 13:05:36.978299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 302/612] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:36 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:37.181784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-27a77722-4a70-482b-983f-e9e2822fe898 req-9e08f1a6-ffb0-4f62-9518-fcb7b9cfa7e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f", "name": "network-vif-deleted", "tag": "e027552b-2e69-458a-acf3-a19901e63f06"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:05:37.190605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-27a77722-4a70-482b-983f-e9e2822fe898 req-9e08f1a6-ffb0-4f62-9518-fcb7b9cfa7e7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:37.191150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-27a77722-4a70-482b-983f-e9e2822fe898 req-9e08f1a6-ffb0-4f62-9518-fcb7b9cfa7e7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:37.202552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-27a77722-4a70-482b-983f-e9e2822fe898 req-9e08f1a6-ffb0-4f62-9518-fcb7b9cfa7e7 service nova] Creating event network-vif-deleted:e027552b-2e69-458a-acf3-a19901e63f06 for instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f on hv-cinder-maste May 23 13:05:37.207567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-27a77722-4a70-482b-983f-e9e2822fe898 req-9e08f1a6-ffb0-4f62-9518-fcb7b9cfa7e7 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088458 May 23 13:05:37.212227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 311/613] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:05:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:05:37.983560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:37.984090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:37.990411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:37.990934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:37.991691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:37.992126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:37.997515 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:37.998229 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:38.004338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:38.004787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:38.005549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:38.006011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:38.024569 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:38.025017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:38.025472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lazy-loading 'fault' on Instance uuid 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:05:38.094513 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c6147fed25a4212b6175aacc460943d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccd27933078b44b48323995e98b21318', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:38.103123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3af5ae06-0654-44d4-bbc8-5426a5de7c08 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 200 len: 1309 microversion: 2.1 time: 0.123335 May 23 13:05:38.104241 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 303/614] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:37 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 1309 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:05:38.164268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:38.172326 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:38.173818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:38.174992 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0e49d7ed-4f59-4007-9731-aaf40eb6917a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.182948 May 23 13:05:38.175558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 312/615] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:37 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:05:39.124834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:39.125450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] [instance: 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:39.131990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:39.132442 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:39.133734 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:39.134293 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:39.147483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] HTTP exception thrown: Instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f could not be found. May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Request method failure captured: May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f HTTP/1.1 May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4XXq-uWfwZEWkYRO-aadJr-HTlQ9NZcOW6Bov--ZXryPok-tdln_HuTrEB7UR2xv4UQ1qA1H9KBdvAWr1u9A3z0zvgPxPnN9xWr9hIOZD-6T_umRMjW12Z7eCp-f-Z4e1lg4HjRoMfcw1Vy_eAxJrT1Vp6JrL2UCjORc-kpwKctJTA May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: ccd27933078b44b48323995e98b21318 May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesSnapshotTestJSON-154686593 May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:05:39.149754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_ccd27933078b44b48323995e98b21318"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/ccd27933078b44b48323995e98b21318"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/ccd27933078b44b48323995e98b21318"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/ccd27933078b44b48323995e98b21318"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesSnapshotTestJSON-154686593 May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: ccd27933078b44b48323995e98b21318 May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-154686593 May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesSnapshotTestJSON-154686593-project May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 5c6147fed25a4212b6175aacc460943d May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesSnapshotTestJSON-154686593-project May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f could not be found. May 23 13:05:39.151296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f'} May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f could not be found. May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:05:39.152052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f could not be found. May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f could not be found. May 23 13:05:39.153468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:05:39.154857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] Returning 404 to user: Instance 9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:05:39.154857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c7341596-e1b8-4d5b-b1d5-9a2ac2744b99 tempest-VolumesSnapshotTestJSON-154686593 tempest-VolumesSnapshotTestJSON-154686593-project] 10.222.0.30 "GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f" status: 404 len: 111 microversion: 2.1 time: 0.030212 May 23 13:05:39.154857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 304/616] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:05:39 2022] GET /compute/v2.1/servers/9dbdf1fe-29a7-4b08-b4bb-0ab47f4b022f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:05:39.193274 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:39.193936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:39.200963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:39.201413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:39.202221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:39.202659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:39.330052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:39.338593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:39.340082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:39.341230 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0c58ac5d-75f7-4c93-9aec-8fbaa3250be6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.151782 May 23 13:05:39.342085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 313/617] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:39 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:40.360365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:40.365010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:40.372520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:40.372963 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:40.373717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:40.374177 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:40.504952 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:40.513298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:40.514808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:40.515972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f29cc797-b133-4372-be32-daf66ab18848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.159563 May 23 13:05:40.516514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 305/618] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:40 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:41.534632 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:41.535233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:41.542343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:41.542781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:41.546051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:41.546051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:41.670256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:41.678025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:41.679511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:41.680881 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cd00ccc2-933a-43ba-9c58-27e3f5b57031 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.149924 May 23 13:05:41.681444 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 314/619] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:41 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:42.700635 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:42.701241 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:42.707712 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:42.708152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:42.708904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:42.709335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:42.840405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:42.848481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:42.850028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:42.851198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a129e918-0fce-42dd-9c0c-f64efc57365a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.156301 May 23 13:05:42.851747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 306/620] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:42 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:43.870076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:43.870708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:43.877078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:43.877517 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:43.878292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:43.878722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:44.006999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:44.014962 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:44.016455 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:44.017595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f751b676-cb25-4c29-8d08-b74bee814408 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.151508 May 23 13:05:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 315/621] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:43 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:45.036554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:45.037083 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:45.044653 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:45.045054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:45.045812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:45.046271 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:45.178189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:45.186552 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:45.188053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:45.189172 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3cc38c44-5ed6-4d5e-b071-d1d668609fd6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.156585 May 23 13:05:45.189715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 307/622] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:45 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:46.207297 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:46.207895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:46.215013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:46.215459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:46.216206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:46.216635 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:46.342755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:46.350584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:46.352060 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:46.354416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f0a18221-c763-4201-b29a-7c3def57e17d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.150960 May 23 13:05:46.354977 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 316/623] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:46 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:47.373306 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:47.373934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:47.380412 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:47.380802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:47.381589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:47.382087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:47.396959 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:47.512349 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:47.520167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:47.521925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:47.523066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a7bd8c8d-8980-4118-900f-13032fb29502 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.153663 May 23 13:05:47.523617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 308/624] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:47 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:48.541861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:48.542492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:48.549008 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:48.549457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:48.550253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:48.550686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:48.686454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:48.693263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:48.694785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:48.696040 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-69e426d3-7f93-4652-a756-a5bd0ce1ea37 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.158480 May 23 13:05:48.698372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 317/625] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:48 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:05:49.716581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:49.717189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:49.723798 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:49.724246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:49.725003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:49.725440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:49.864585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:49.873386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:49.875085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:49.876296 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-732f79b2-1cfe-4505-9f91-1e5f758d97d1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.164779 May 23 13:05:49.876853 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 309/626] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:49 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:50.645392 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-6449b33e-9339-4ff9-b3d2-d78b6ab48482 req-74cb6a53-b264-49ba-9c12-a8069c97e087 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "name": "network-vif-plugged", "status": "completed", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:05:50.654150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-6449b33e-9339-4ff9-b3d2-d78b6ab48482 req-74cb6a53-b264-49ba-9c12-a8069c97e087 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:50.654593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-6449b33e-9339-4ff9-b3d2-d78b6ab48482 req-74cb6a53-b264-49ba-9c12-a8069c97e087 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:50.665246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-6449b33e-9339-4ff9-b3d2-d78b6ab48482 req-74cb6a53-b264-49ba-9c12-a8069c97e087 service nova] Creating event network-vif-plugged:cee5aa36-f69a-432b-9f71-0cc93809d9a9 for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 on hv-cinder-maste May 23 13:05:50.670579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-6449b33e-9339-4ff9-b3d2-d78b6ab48482 req-74cb6a53-b264-49ba-9c12-a8069c97e087 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082088 May 23 13:05:50.671164 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 318/627] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:05:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:05:50.894552 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:50.895153 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:50.901865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:50.902333 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:50.903082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:50.903516 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:51.039579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:51.048190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:51.049705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:51.050895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5d73fdfe-a57b-4dee-b1c1-dc5d40b8c746 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1744 microversion: 2.20 time: 0.160216 May 23 13:05:51.051488 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 310/628] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:50 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1744 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:52.070622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:52.071378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:52.077629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:52.078122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:52.078886 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:52.079323 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:52.207544 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:52.216209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:52.217719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:52.218887 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3814be48-b428-4313-b9b3-d1067860ef45 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.153313 May 23 13:05:52.219441 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 319/629] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:52 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:53.244321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:53.245106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:53.249726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:53.253612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:53.254185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:53.259408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:53.259408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:53.403127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:53.414539 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:53.416807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:53.418283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4840c4a0-135a-4870-b9bd-b3f5cdefaf77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.179004 May 23 13:05:53.418907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 311/630] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:53 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:54.438022 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:54.439666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:54.448873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:54.449613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:54.450702 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:54.451368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:54.479775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:54.596505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:54.604761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:54.606575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:54.607736 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-70f9b2a3-d72c-4052-ac0c-00706eb6cf14 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.173983 May 23 13:05:54.608325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 320/631] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:54 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:55.626977 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:55.627603 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:55.634520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:55.635011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:55.689425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:55.689425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:55.771197 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:55.780375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:55.781908 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:55.783047 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a796b7ef-fc07-4754-9d59-f35f0a96b04b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.160303 May 23 13:05:55.783825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 312/632] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:55 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:56.800808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:56.801499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:56.804274 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=36 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:05:56.810976 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:56.817453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:56.817453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:56.817453 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:57.204143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:57.218883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:57.222426 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:57.224131 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-711ed888-b4e6-40b2-8551-1910cc767a77 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.426984 May 23 13:05:57.225010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 321/633] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:56 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:05:58.248304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:58.249301 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:58.256857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:58.259124 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:58.259960 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:58.260395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:58.419207 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:58.427718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:58.429249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:58.430432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-453d8abb-0abf-49d9-a766-4a4837461922 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.188898 May 23 13:05:58.437917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 313/634] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:58 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:05:59.454559 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:05:59.455295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:05:59.462691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:59.463775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:59.464663 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:05:59.465140 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:05:59.618168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:59.628983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:59.631523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:05:59.633059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a90b5139-54a9-4848-8a74-285246664ceb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.183249 May 23 13:05:59.640169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 322/635] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:05:59 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:00.655441 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:00.656384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:00.662901 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:00.663630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:00.667589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:00.668723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:00.835261 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:00.850964 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:00.852632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:00.854748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2ff80d03-ee3f-43d2-80cf-ac1eb7c0ee11 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.205039 May 23 13:06:00.858585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 314/636] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:00 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:01.876106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:01.876718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:01.884522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:01.885563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:01.886446 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:01.886954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:02.022533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:02.030769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:02.032246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:02.033384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4d89fccb-3f22-40eb-af59-32a4d1a79f65 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.161422 May 23 13:06:02.033956 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 323/637] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:01 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:03.051996 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:03.052563 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:03.059811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:03.060987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:03.062185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:03.062987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:03.202189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:03.211873 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:03.213452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:03.214743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d25169aa-de2b-41f5-baf4-752819a580a9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.166676 May 23 13:06:03.215313 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 315/638] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:03 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:04.233648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:04.234275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:04.240655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:04.241096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:04.241843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:04.242305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:04.373438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:04.381222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:04.382762 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:04.383902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-17912c39-dc4d-48b0-a528-8e996fb619d9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1739 microversion: 2.20 time: 0.154644 May 23 13:06:04.384454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 324/639] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:04 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1739 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:05.402150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:05.403440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:05.410208 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:05.410724 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:05.413413 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:05.413413 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:05.591833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:05.599645 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:05.601127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:05.602285 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9394deb3-303f-4127-8fd6-96bbf3c1587e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1721 microversion: 2.20 time: 0.204375 May 23 13:06:05.602829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 316/640] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:05 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1721 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:06.622271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:06.623136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:06.629523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:06.630001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:06.630752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:06.631177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:06.760437 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:06.769074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:06.770605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:06.772015 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7d9228d3-9315-43d6-950a-b71b6dfd05ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1721 microversion: 2.20 time: 0.154528 May 23 13:06:06.772562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 325/641] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:06 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:07.789701 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:07.790331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:07.797189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:07.797668 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:07.798466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:07.798939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:07.927678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:07.935922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:07.937437 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:07.938600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-50a695ba-31e2-4fa8-a036-d7815dcc3612 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1724 microversion: 2.20 time: 0.152824 May 23 13:06:07.939145 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 317/642] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:07 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:07.956623 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e9a2d9e1-1137-4e4e-a1da-d6db1bcbc8a6"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:07.957586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:07.964002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:07.964473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:07.965227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:07.965658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:08.782341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-37df8fd5-6656-428d-8aee-abd00519222e req-5b621a7d-3d66-4048-ae67-11bd3ae8d095 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "name": "network-vif-unplugged", "status": "completed", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:08.791912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-37df8fd5-6656-428d-8aee-abd00519222e req-5b621a7d-3d66-4048-ae67-11bd3ae8d095 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:08.791912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-37df8fd5-6656-428d-8aee-abd00519222e req-5b621a7d-3d66-4048-ae67-11bd3ae8d095 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:08.801964 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.compute.server_external_events [req-37df8fd5-6656-428d-8aee-abd00519222e req-5b621a7d-3d66-4048-ae67-11bd3ae8d095 service nova] Unable to find a host for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515. Dropping event network-vif-unplugged {{(pid=130046) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} May 23 13:06:08.802819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-37df8fd5-6656-428d-8aee-abd00519222e req-5b621a7d-3d66-4048-ae67-11bd3ae8d095 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.024256 May 23 13:06:08.803433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 318/643] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:06:08 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 25 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) May 23 13:06:09.068006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d97b3af1-3a5e-41d3-b873-7818fbf55cbd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.115257 May 23 13:06:09.068811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 326/644] 10.222.0.30 () {62 vars in 1363 bytes} [Mon May 23 13:06:07 2022] POST /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments => generated 188 bytes in 1117 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 13:06:09.184054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:09.184614 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:09.190969 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:09.191411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:09.194705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:09.194705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:09.432246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-89328d73-d8ca-4f53-89f5-acee04e1793c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/action" status: 202 len: 0 microversion: 2.20 time: 0.251997 May 23 13:06:09.432857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 319/645] 10.222.0.30 () {62 vars in 1318 bytes} [Mon May 23 13:06:09 2022] POST /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/action => generated 0 bytes in 253 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 13:06:09.441682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:09.442327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:09.448537 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:09.448981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:09.449752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:09.450230 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:09.585107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:09.593740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:09.595253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:09.596381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-91970810-8dd8-40b0-8369-741a9a785bab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1750 microversion: 2.20 time: 0.158627 May 23 13:06:09.596933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 327/646] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:09 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1750 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:10.615253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:10.615857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:10.622567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:10.623011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:10.623876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:10.624303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:10.754209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:10.762369 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:10.763849 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:10.765638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a2ead10d-a903-44aa-b2b3-8b91ee6539fc tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1750 microversion: 2.20 time: 0.154752 May 23 13:06:10.766212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 320/647] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:10 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1750 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:11.783845 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:11.784456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:11.791145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:11.791594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:11.792363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:11.792803 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:11.938637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:11.947155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:11.948575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:11.949713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7485dac3-3f75-4b5d-a4f3-057e7cd187e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1750 microversion: 2.20 time: 0.170515 May 23 13:06:11.950303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 328/648] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:11 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1750 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:12.969015 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:12.969620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:12.976231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:12.976684 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:12.977451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:12.977910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:13.177172 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:13.187330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:13.189372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:13.190694 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b9cd3a9d-422b-413d-8d51-4943f11ac0b3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1750 microversion: 2.20 time: 0.226189 May 23 13:06:13.191561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 321/649] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:12 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1750 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:14.209589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:14.210445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:14.217524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:14.217992 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:14.218771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:14.219203 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:14.370003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:14.379205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:14.380821 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:14.381994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ed286bc9-5c1c-4211-95e3-f030bc4463f0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1750 microversion: 2.20 time: 0.176347 May 23 13:06:14.382541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 329/650] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:14 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1750 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:15.401290 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:15.401902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:15.408424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:15.408848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:15.409606 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:15.410065 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:15.539820 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:15.548422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:15.549821 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:15.551000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7a9e3a9a-da8a-4f7f-aa97-30aed6f8d91a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1752 microversion: 2.20 time: 0.153510 May 23 13:06:15.551546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 322/651] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:15 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:16.147099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c27f6163-8522-414a-aa9b-20eb17ae7934"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:16.148199 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:16.154268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:16.154723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:16.156054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:16.156557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:16.570843 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:16.571735 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:16.578662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:16.579101 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:16.579858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:16.580328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:16.726206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:16.735099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:16.739400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:16.739400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e5db8cb2-c8a5-47f2-99d5-23ab522e8a22 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1752 microversion: 2.20 time: 0.171862 May 23 13:06:16.739400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 323/652] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:16 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1752 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:16.943415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-01ed0105-040a-4677-90c1-e0e72328d9b2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.021604 May 23 13:06:16.944012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 330/653] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:06:15 2022] POST /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments => generated 194 bytes in 1024 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:17.757459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:17.758081 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:17.764772 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:17.765474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:17.766264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:17.766697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:17.769255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:17.900866 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:17.910446 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:17.912183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:17.913417 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-10e627a5-5557-4147-8035-63ecd2f34c6f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.160196 May 23 13:06:17.914230 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 324/654] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:17 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:18.934580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:18.936035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:18.944225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:18.944942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:18.945781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:18.946175 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:19.088604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:19.097927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:19.099443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:19.100658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a61e7054-487f-4a66-8e78-b99a773b5914 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.170892 May 23 13:06:19.101203 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 331/655] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:18 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:20.119680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:20.120302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:20.128144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:20.128597 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:20.129379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:20.129820 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:20.270878 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:20.279571 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:20.281197 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:20.282392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-83c03a49-f8fc-4581-82bc-7785ffa84d79 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.167443 May 23 13:06:20.282954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 325/656] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:20 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:21.300879 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:21.301484 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:21.308201 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:21.308650 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:21.309450 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:21.310101 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:21.447762 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:21.457074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:21.458580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:21.459728 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2242db40-7315-4a27-8204-4170c84c0b0d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.162818 May 23 13:06:21.460473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 332/657] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:21 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:22.480052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:22.480052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:22.487025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:22.487479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:22.488269 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:22.488706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:22.651255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:22.654253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:22.656389 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:22.657762 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-55594134-8d8c-4aa9-a807-6c71d652c667 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.182864 May 23 13:06:22.658357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 326/658] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:22 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:23.676174 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:23.676796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:23.683784 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:23.684226 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:23.685023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:23.685424 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:23.689520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=4,nova_cell1:UPDATE=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:23.831697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:23.841740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:23.843473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:23.844674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-21626fc0-ca67-4af4-afba-ab82da824e23 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.172493 May 23 13:06:23.845282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 333/659] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:23 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:24.862998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:24.863690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:24.866697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=34 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:24.871615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:24.872070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:24.872870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:24.873290 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:25.051619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:25.061003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:25.063079 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:25.064247 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0025c82e-66ca-4e13-94f2-a9c7fa5756fa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.205827 May 23 13:06:25.065070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 327/660] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:24 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:26.082790 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:26.084002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:26.091249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:26.091689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:26.092482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:26.092917 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:26.224076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:26.232567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:26.234951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:26.236211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4393a140-2a06-427c-a94f-7e89543fad2d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.157730 May 23 13:06:26.237188 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 334/661] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:26 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:27.255181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:27.255932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:27.263023 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:27.263825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:27.265443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:27.266259 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:27.418567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:27.429922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:27.429922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:27.430765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d7440502-9b38-4b17-a9dc-c1a35c902623 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.179571 May 23 13:06:27.438333 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 328/662] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:27 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:28.091275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:28.091824 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:28.093707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=28 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:28.098691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.099241 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.100073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.100510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.126822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d81410d0-1cd1-4190-8f91-d0d883670339 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/ips" status: 200 len: 150 microversion: 2.1 time: 0.156837 May 23 13:06:28.127563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 335/663] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:06:27 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/ips => generated 150 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:28.138231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "34a6495d-3741-497d-bb5d-d7dd465d95ff"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:28.140059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:28.146118 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.146825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.147590 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.148025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.228166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Invalid volume: volume 34a6495d-3741-497d-bb5d-d7dd465d95ff is already attached to instances: e01afbdf-8ced-4f91-b14c-b87b2fcf1506 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: POST /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/os-volume_attachments HTTP/1.1 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 74 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:06:28.229914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {"volumeAttachment": {"volumeId": "34a6495d-3741-497d-bb5d-d7dd465d95ff"}} May 23 13:06:28.231524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Invalid volume: volume 34a6495d-3741-497d-bb5d-d7dd465d95ff is already attached to instances: e01afbdf-8ced-4f91-b14c-b87b2fcf1506 May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'server_id': 'adac7929-c83b-4f7b-9b97-1b7a48da67a2', 'body': {'volumeAttachment': {'volumeId': '34a6495d-3741-497d-bb5d-d7dd465d95ff'}}} May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 34a6495d-3741-497d-bb5d-d7dd465d95ff is already attached to instances: e01afbdf-8ced-4f91-b14c-b87b2fcf1506 May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:28.232467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 34a6495d-3741-497d-bb5d-d7dd465d95ff is already attached to instances: e01afbdf-8ced-4f91-b14c-b87b2fcf1506 May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:28.234123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 34a6495d-3741-497d-bb5d-d7dd465d95ff is already attached to instances: e01afbdf-8ced-4f91-b14c-b87b2fcf1506 May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 400 to user: Invalid volume: volume 34a6495d-3741-497d-bb5d-d7dd465d95ff is already attached to instances: e01afbdf-8ced-4f91-b14c-b87b2fcf1506 {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ae9c8255-817c-4407-b608-3141af203aa5 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.098581 May 23 13:06:28.235588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 329/664] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:06:28 2022] POST /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2/os-volume_attachments => generated 174 bytes in 100 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 13:06:28.242469 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:28.242969 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:28.249952 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.250399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.251158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.251672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.277400 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:06:28.450124 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:28.450761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:28.459039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.459504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.460368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.460825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.539956 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e69e1ef7-8b63-46b3-8344-4a4eeeedda31 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 204 len: 0 microversion: 2.1 time: 0.300422 May 23 13:06:28.539956 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 336/665] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:06:28 2022] DELETE /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:06:28.547996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:28.548680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:28.554677 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.555119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.555873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:28.556313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:28.592117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:28.601746 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:28.604700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:28.605966 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dccaf79f-af82-4c56-921a-240099e85b08 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.159697 May 23 13:06:28.614495 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 330/666] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:28 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:28.693063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:28.700548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6b0c0de0-964a-4bd8-a574-286a672bc7ff tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.156505 May 23 13:06:28.701108 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 337/667] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:28 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:29.624632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:29.625243 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:29.631392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:29.631836 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:29.632595 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:29.633020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:29.720732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:29.721339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:29.727896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:29.728344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:29.729146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:29.729572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:29.778899 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:29.787167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:29.788667 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:29.789801 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cf62fa4c-89ea-411f-9ad5-517b0ed04896 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.169321 May 23 13:06:29.790374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 331/668] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:29 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:29.862430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:29.869916 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-edfc8fd8-1471-42dc-b607-131f792e04c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.153484 May 23 13:06:29.870477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 338/669] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:29 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:30.807944 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:30.808554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:30.815177 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:30.815622 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:30.816376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:30.816810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:30.887586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:30.888175 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:30.895425 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:30.895866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:30.897234 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:30.897334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:30.978008 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:30.986325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:30.987807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:30.988946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ec2de93a-446c-46c7-83e6-aaf53fb4c3cf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.184863 May 23 13:06:30.989487 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 332/670] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:30 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:31.037731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:31.045052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-10ee10e7-5eda-45d1-9e5a-17c6d3ccc9ee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.161343 May 23 13:06:31.045615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 339/671] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:30 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:32.007279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:32.007807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:32.014272 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:32.014718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:32.016099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:32.016533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:32.063395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:32.064035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:32.070584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:32.071034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:32.071794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:32.072267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:32.201664 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:32.210522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:32.212026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:32.213166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8313e902-940a-401e-9630-d44942e2d698 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.209957 May 23 13:06:32.213718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 333/672] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:32 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:32.228997 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:32.236779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e778196d-cefb-4a66-a618-bcf828a35c99 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.177946 May 23 13:06:32.237337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 340/673] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:32 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:33.232424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:33.233027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:33.239657 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:33.240098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:33.240842 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:33.241272 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:33.253712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:33.254355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:33.260727 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:33.261174 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:33.261958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:33.262402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:33.432504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:33.441537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:33.443059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:33.444209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8f6f815b-c9ad-4800-a8ba-252d8f039ab6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.215721 May 23 13:06:33.444761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 334/674] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:33 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:33.458517 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:33.467640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9cac018b-ed48-4d62-8fc4-73e88be07a48 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.216836 May 23 13:06:33.467640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 341/675] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:33 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:34.464936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:34.465539 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:34.472971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:34.473415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:34.474357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:34.475335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:34.483625 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:34.487580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:34.491697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:34.492347 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:34.493130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:34.493570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:34.698387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:34.700787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:34.707390 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:34.708643 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d5ce0062-faa3-448e-b9d7-6494d6ae4247 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.229141 May 23 13:06:34.708889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:34.709210 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 342/676] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:34 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:34.710122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-73ef0b08-a502-4c3e-aa51-1a8cc26ff223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.249255 May 23 13:06:34.710670 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 335/677] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:34 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:35.725539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:35.726166 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:35.728263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:35.728877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:35.732589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:35.736896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:35.736896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:35.736896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:35.737206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:35.737206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:35.737206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:35.737631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:36.051018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:36.060535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7c49b2dd-5238-4973-90a3-18450314d719 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1681 microversion: 2.1 time: 0.338810 May 23 13:06:36.061914 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 343/678] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:35 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1681 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:36.063815 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:36.073521 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:36.075470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:36.076883 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-baf2168d-bdb3-4e37-bbe4-b583ac3ffd24 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.352499 May 23 13:06:36.078845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 336/679] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:35 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:36.120546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:36.121479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:36.127740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:36.128447 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:36.129226 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:36.129665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:36.153193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lazy-loading 'flavor' on Instance uuid 462787a7-152e-4a6a-9c26-740f68c60a77 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:06:36.262934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-ba6d3d03-a23a-499c-b5da-4c4e219f8d8a req-b6ab7589-4183-439d-80ca-3e51c0dbcf64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "adac7929-c83b-4f7b-9b97-1b7a48da67a2", "name": "network-vif-deleted", "tag": "7f1679f8-1ca4-4c8a-8075-674c69981de0"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:36.272294 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-ba6d3d03-a23a-499c-b5da-4c4e219f8d8a req-b6ab7589-4183-439d-80ca-3e51c0dbcf64 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:36.272725 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-ba6d3d03-a23a-499c-b5da-4c4e219f8d8a req-b6ab7589-4183-439d-80ca-3e51c0dbcf64 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:36.285375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-ba6d3d03-a23a-499c-b5da-4c4e219f8d8a req-b6ab7589-4183-439d-80ca-3e51c0dbcf64 service nova] Creating event network-vif-deleted:7f1679f8-1ca4-4c8a-8075-674c69981de0 for instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 on hv-cinder-maste May 23 13:06:36.290662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-ba6d3d03-a23a-499c-b5da-4c4e219f8d8a req-b6ab7589-4183-439d-80ca-3e51c0dbcf64 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031692 May 23 13:06:36.291421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 337/680] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:06:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:06:36.554169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-932cd33a-be4d-4a04-9aa4-ed40cd0db16a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/action" status: 200 len: 32 microversion: 2.1 time: 0.437503 May 23 13:06:36.554883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 344/681] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:06:36 2022] POST /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/action => generated 32 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:06:36.565338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:36.566066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:36.572938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:36.573397 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:36.574459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:36.574910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:36.720558 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:36.728594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b3b328b2-3530-426f-a133-24fe7da38698 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.168241 May 23 13:06:36.738102 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 338/682] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:36 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:37.078941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:37.080276 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:37.087025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.087468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.093145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.094065 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.100050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:37.105491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:37.108900 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.110315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.111264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.111860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.130151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.130761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.131514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lazy-loading 'fault' on Instance uuid adac7929-c83b-4f7b-9b97-1b7a48da67a2 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:06:37.221147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:37.231667 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-71ea512f-08ea-4841-bf7f-a3c26c7ed2dc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 200 len: 1331 microversion: 2.1 time: 0.157561 May 23 13:06:37.233086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 345/683] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:37 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 1331 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:37.352112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:37.361463 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:37.369938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:37.371571 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0a1ce2dd-e90a-4048-a741-3b9b9139833e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.275273 May 23 13:06:37.372628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 339/684] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:37 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:37.746628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:37.747232 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:37.753862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.754333 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.755086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:37.755516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:37.759507 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:37.884945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:37.893431 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-214406e0-2b12-445a-8f8d-f2b6e6bca135 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.150711 May 23 13:06:37.894018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 346/685] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:37 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:38.249147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:38.249919 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:38.256366 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:38.256810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:38.257556 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:38.258014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:38.271427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 HTTP/1.1 May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:06:38.273315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:06:38.274807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'adac7929-c83b-4f7b-9b97-1b7a48da67a2'} May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:06:38.275545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:06:38.276810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:38.277953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:06:38.277953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a5c0fd52-90d7-46f4-929f-cd5890578f1b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 404 len: 111 microversion: 2.1 time: 0.029443 May 23 13:06:38.277953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 340/686] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:38 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:06:38.284530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0c742879-3817-4f95-879e-40a5e69fe665 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '.version_select at 0x7f9b1c03d790>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:38.287866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR oslo.messaging._drivers.impl_rabbit [None req-0c742879-3817-4f95-879e-40a5e69fe665 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [91422227-95e9-4650-ad38-b478e8ff007f] AMQP server on 10.222.0.30:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 13:06:38.494375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:38.495062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:38.502446 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:38.502959 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:38.503789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:38.504295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:38.638547 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:38.648765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:38.650334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:38.651506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-22f6806e-3874-4125-b946-96a93bcb31ab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.265004 May 23 13:06:38.652057 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 341/687] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:38 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:38.911482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:38.912220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:38.918840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:38.919297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:38.920086 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:38.920528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:38.924167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:39.057029 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:39.065573 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-54d600a3-0b93-4c77-9fa2-c43fc7bbec26 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.158206 May 23 13:06:39.066155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 342/688] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:38 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:39.295559 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [None req-0c742879-3817-4f95-879e-40a5e69fe665 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [91422227-95e9-4650-ad38-b478e8ff007f] Reconnected to AMQP server on 10.222.0.30:5672 via [amqp] client with port 41560. May 23 13:06:39.368690 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0c742879-3817-4f95-879e-40a5e69fe665 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1093406334" status: 202 len: 0 microversion: 2.1 time: 1.088712 May 23 13:06:39.369260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 347/689] 10.222.0.30 () {58 vars in 1220 bytes} [Mon May 23 13:06:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1093406334 => generated 0 bytes in 1090 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:06:39.669824 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:39.670522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:39.675510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:06:39.675743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:06:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:06:39.681263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:39.682131 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:39.683131 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:39.683719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:39.827170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:39.836101 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:39.837666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:39.838907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8644105b-5075-4c94-8a2c-23c8a34e31d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.173084 May 23 13:06:39.839608 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 343/690] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:39 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:40.083401 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:40.084004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:40.089907 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.090371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.091123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.091552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.166206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:40.166836 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:40.174447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.175003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.175852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.176362 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.239445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:40.247301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e2d607a0-b3ed-429e-ad39-18a37f0c9883 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.167815 May 23 13:06:40.247864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 348/691] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:40 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:40.259337 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lazy-loading 'services' on Instance uuid e01afbdf-8ced-4f91-b14c-b87b2fcf1506 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:06:40.463940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b8408c97-0fe2-46ee-ae16-6018b60dead0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments/34a6495d-3741-497d-bb5d-d7dd465d95ff" status: 202 len: 0 microversion: 2.1 time: 0.301543 May 23 13:06:40.464533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 344/692] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:06:40 2022] DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments/34a6495d-3741-497d-bb5d-d7dd465d95ff => generated 0 bytes in 303 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:06:40.473050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:40.474027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:40.480535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.480961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.481725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.482187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.511981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d55427c5-687c-45dc-9dfb-5a1950edc6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043011 May 23 13:06:40.512473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 349/693] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:06:40 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:40.860153 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:40.860853 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:40.867158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.867607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.868370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:40.868809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:40.994537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:41.003049 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:41.004523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:41.005651 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-76b34bda-10b2-43be-a69c-168223af5e60 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.150883 May 23 13:06:41.006249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 345/694] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:40 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:41.266036 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:41.266531 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:41.272717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:41.273156 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:41.273922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:41.274368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:41.402483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:41.410915 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-24f12fdd-d331-4faf-86eb-0d05a7304cd3 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.149261 May 23 13:06:41.411471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 350/695] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:41 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:41.523773 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:41.524671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:41.530576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:41.531073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:41.531822 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:41.532255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:41.562810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c622a8cf-03af-481a-ae5b-8ddcebc5d6c2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042544 May 23 13:06:41.563436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 346/696] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:06:41 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:42.024143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:42.024750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:42.031527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:42.031974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:42.032732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:42.033166 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:42.161218 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:42.169588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:42.171180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:42.173097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e7df720d-d0ad-4452-9d30-3c44322b7afd tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.153135 May 23 13:06:42.173655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 351/697] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:42 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:42.428486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:42.429093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:42.435480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:42.435920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:42.436669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:42.437190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:42.563683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:42.572776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-209eeb14-2313-4ca3-9fd0-0594daec6e11 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.148208 May 23 13:06:42.573494 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 347/698] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:42 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:42.574610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:42.575654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:42.582258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:42.582948 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:42.583795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:42.584645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:42.613647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7dcb9e55-4c30-4c2d-bd59-acc66d46679d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043193 May 23 13:06:42.614233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 352/699] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:06:42 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:06:43.191445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:43.192070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:43.198120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.198498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.199242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.199674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.331404 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:43.340594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:43.342547 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:43.343954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7091630e-f09e-43ba-a44a-1188c21c184e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.157087 May 23 13:06:43.344505 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 348/700] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:43 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:43.590393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:43.591120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:43.597902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.598368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.599122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.599558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.626152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:43.627246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:43.633600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.634084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.634845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.635284 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.667309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-77b6e451-525c-4d9c-8841-cb2219321be2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045820 May 23 13:06:43.667879 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 349/701] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:06:43 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 23 13:06:43.729325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:43.729818 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:43.731505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:43.736631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.737079 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.737840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:43.738303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:43.740158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6a89a761-5a86-405c-a5bc-1df91f6109b0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.153387 May 23 13:06:43.740731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 353/702] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:43 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:43.762632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:06:44.065227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d6be7e66-2211-42e1-bf6e-c7cc9bf3d4fc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 204 len: 0 microversion: 2.1 time: 0.339396 May 23 13:06:44.065850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 350/703] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:06:43 2022] DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 13:06:44.073938 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:44.074542 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:44.082004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.092816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.092816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.092816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.221855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:44.229361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c07de4ea-b3a3-4fae-a496-d770b1043c64 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1681 microversion: 2.1 time: 0.159294 May 23 13:06:44.229935 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 354/704] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:44 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:44.362014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:44.362715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:44.368639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.369093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.369852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.370316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.501017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:44.509581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:44.511111 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:44.512888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8dcd1be9-0551-43b2-8ecc-367d165f40aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.154937 May 23 13:06:44.513856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 351/705] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:44 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:44.757935 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:44.758540 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:44.764922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.765967 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.766722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.767158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.901379 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:44.910695 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b12298c1-9a8f-42c8-a7b6-464641b86bbe tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.156587 May 23 13:06:44.911279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 355/706] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:44 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:44.935037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-c2151183-1ca9-4ac5-b708-90ff3d2e4ecb req-e7e9cff5-0285-4fd5-beb8-8462adff4578 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "name": "network-vif-plugged", "status": "completed", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:44.944097 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-c2151183-1ca9-4ac5-b708-90ff3d2e4ecb req-e7e9cff5-0285-4fd5-beb8-8462adff4578 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:44.946110 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-c2151183-1ca9-4ac5-b708-90ff3d2e4ecb req-e7e9cff5-0285-4fd5-beb8-8462adff4578 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:44.955268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-c2151183-1ca9-4ac5-b708-90ff3d2e4ecb req-e7e9cff5-0285-4fd5-beb8-8462adff4578 service nova] Creating event network-vif-plugged:cee5aa36-f69a-432b-9f71-0cc93809d9a9 for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 on hv-cinder-maste May 23 13:06:44.960689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-c2151183-1ca9-4ac5-b708-90ff3d2e4ecb req-e7e9cff5-0285-4fd5-beb8-8462adff4578 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029208 May 23 13:06:44.961273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 352/707] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:06:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:45.247773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:45.248337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:45.254894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:45.255347 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:45.256104 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:45.256538 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:45.383384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:45.391268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1a617073-d57d-4cb9-9879-6bfbad6dc84c tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1681 microversion: 2.1 time: 0.147407 May 23 13:06:45.391840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 356/708] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:45 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1681 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:45.533729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:45.534649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:45.540856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:45.541384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:45.542260 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:45.542747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:45.825638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:45.835062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:45.836663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:45.837820 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d0491579-f265-4814-b46c-1b105ebda8e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.309032 May 23 13:06:45.838403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 353/709] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:45 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:45.929582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:45.930253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:45.936688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:45.937759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:45.938449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:45.938880 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:46.115622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:46.123472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4b42e70f-bfa2-4a79-bb73-fbf78a316d42 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.198785 May 23 13:06:46.124021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 357/710] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:45 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:46.408922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:46.409480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:46.415789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:46.416228 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:46.416987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:46.417416 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:46.717688 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:46.726396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d7059875-e55f-45a4-9f16-9e65edef0448 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1614 microversion: 2.1 time: 0.321845 May 23 13:06:46.726965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 354/711] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:46 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1614 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:46.729914 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-e52b874a-c251-4adb-b469-99b77d12695d req-c1c9d532-60e4-45e1-90b2-e2dac122a6c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e01afbdf-8ced-4f91-b14c-b87b2fcf1506", "name": "network-vif-deleted", "tag": "97ab468b-d2e1-4b56-98d6-23dfeb309dc3"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:46.740062 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-e52b874a-c251-4adb-b469-99b77d12695d req-c1c9d532-60e4-45e1-90b2-e2dac122a6c3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:46.740677 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-e52b874a-c251-4adb-b469-99b77d12695d req-c1c9d532-60e4-45e1-90b2-e2dac122a6c3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:46.752087 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-e52b874a-c251-4adb-b469-99b77d12695d req-c1c9d532-60e4-45e1-90b2-e2dac122a6c3 service nova] Creating event network-vif-deleted:97ab468b-d2e1-4b56-98d6-23dfeb309dc3 for instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 on hv-cinder-maste May 23 13:06:46.757509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-e52b874a-c251-4adb-b469-99b77d12695d req-c1c9d532-60e4-45e1-90b2-e2dac122a6c3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031831 May 23 13:06:46.758117 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 358/712] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:06:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:06:46.856102 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:46.856825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:46.863003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:46.863460 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:46.864249 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:46.864663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:46.998963 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:47.008606 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:47.010151 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:47.011387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3074f6e0-5daa-46f0-bb03-91e36e0c1e12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.159262 May 23 13:06:47.012129 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 355/713] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:46 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:47.142393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:47.143088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:47.150313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:47.150762 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:47.151529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:47.151961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:47.295637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:47.305080 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-923fead7-f318-4f88-8950-c8a43199ff17 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.167958 May 23 13:06:47.305682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 359/714] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:47 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:47.744043 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:47.744657 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:47.754074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:47.754074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:47.754074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:47.754074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:47.787722 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:47.788345 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:47.788889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lazy-loading 'fault' on Instance uuid e01afbdf-8ced-4f91-b14c-b87b2fcf1506 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:06:47.832250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:47.834463 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:47.840309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f21a8085-0b73-461d-a24a-ac017ef8fccf tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 200 len: 1331 microversion: 2.1 time: 0.100064 May 23 13:06:47.840867 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 356/715] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:47 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 1331 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:48.030216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:48.031502 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:48.037499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.037986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.038758 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.039225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.172975 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:48.181192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:48.182744 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:48.183892 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dd275909-a43f-4d59-b62f-c29caac3d0c6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1808 microversion: 2.20 time: 0.157490 May 23 13:06:48.184442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 360/716] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:48 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1808 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:06:48.323667 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:48.324280 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:48.331251 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.332761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.332761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.332954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.471121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:48.478906 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cfed3aaf-a9af-47f1-8164-c8d57ad64e14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.159503 May 23 13:06:48.479462 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 357/717] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:48 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:48.858268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:48.860533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:48.868067 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.869095 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.870040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.870690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.894617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 HTTP/1.1 May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:06:48.896273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:06:48.897938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'e01afbdf-8ced-4f91-b14c-b87b2fcf1506'} May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:06:48.898737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:06:48.900003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:48.901114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:06:48.901114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ad7ce98a-6232-4557-b96f-35f56af092a8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 404 len: 111 microversion: 2.1 time: 0.045220 May 23 13:06:48.901114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 358/718] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:48 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:06:48.918479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-12846967-aa84-42e2-a876-8c1c40004bd2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '.version_select at 0x7f9b12ddc310>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:48.939768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:48.940463 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:48.946889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.947442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.948294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:48.948945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:48.975367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:06:49.020749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-12846967-aa84-42e2-a876-8c1c40004bd2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-756628403" status: 202 len: 0 microversion: 2.1 time: 0.109032 May 23 13:06:49.021325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 359/719] 10.222.0.30 () {58 vars in 1217 bytes} [Mon May 23 13:06:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-756628403 => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:06:49.204069 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:49.204669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:49.205847 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c471b8cc-5639-4c73-9d4d-2d23e75712d7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "DELETE /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 204 len: 0 microversion: 2.1 time: 0.389971 May 23 13:06:49.207346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 361/720] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:06:48 2022] DELETE /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:06:49.212136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:49.212580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:49.214195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:49.214633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:49.221298 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:49.222349 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:49.230806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:49.231852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:49.240070 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:49.240948 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:49.525683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:49.534376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:49.535925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:49.539775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e445473b-21db-45cc-ba56-2b6f25cf3e78 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1795 microversion: 2.20 time: 0.337986 May 23 13:06:49.540354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 360/721] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:06:49 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1795 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:06:49.547045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:49.547650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:49.555364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:49.555815 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:49.556586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:49.557027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:49.578679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:49.581368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:49.587413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8b5aa207-5785-4438-8564-195ec0c6f762 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1578 microversion: 2.1 time: 0.376398 May 23 13:06:49.588004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 362/722] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:49 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1578 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:49.766024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:49.774482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0e106872-6ec7-45e6-866e-59320d829b14 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.232874 May 23 13:06:49.775170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 361/723] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:49 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:50.064573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8ea82cab-cb7c-482e-b6a9-e60b03fb60fb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: .version_select at 0x7f9b1c135160>, body: {"keypair": {"name": "tempest-keypair-1934053805", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMLIPJF5C+uIiYrGlJ26Z1yMvvlE+VvtqEjYWQooXpK6hrFPW1vTElMDokY/Ss5X8wab+l9bZcWNWizgf9+N6+GcCc2PJEcYHjfjpo6UfR/xegdj1JUc7v5pTdSBBPhXpQ=="}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:50.079126 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-8ea82cab-cb7c-482e-b6a9-e60b03fb60fb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:06:50.082963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-8ea82cab-cb7c-482e-b6a9-e60b03fb60fb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:06:50.189672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8ea82cab-cb7c-482e-b6a9-e60b03fb60fb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.129168 May 23 13:06:50.190313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 363/724] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:06:50 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:50.604618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:50.605132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:50.611748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:50.612189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:50.612936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:50.613388 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:50.786179 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:50.792373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:50.792980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:50.793709 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cf748bfe-bb21-4116-86b3-8743f3b8a991 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1578 microversion: 2.1 time: 0.193004 May 23 13:06:50.794346 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 362/725] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:50 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1578 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:50.799518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:50.799956 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:50.800714 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:50.801169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:50.942464 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:50.950487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d3b3c573-d817-451c-b1ce-d36a87c76258 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.161954 May 23 13:06:50.951054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 364/726] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:50 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:51.812816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:51.813442 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:51.820727 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:51.821567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:51.822939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:51.823538 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:51.937735 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:51.947801 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dae6eac8-2fcd-48da-88bf-8e15eb9c132d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1511 microversion: 2.1 time: 0.139788 May 23 13:06:51.949004 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 363/727] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:51 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1511 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:51.972899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:51.973520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:51.987923 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-1d4e8722-ddcb-4301-ac34-2819e179f818 req-ccf55743-d146-457c-b2b8-23addacc5ee5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7064ed49-237d-441d-a087-70b195245624", "name": "network-vif-deleted", "tag": "f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:51.996777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:51.997338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:51.998120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:51.998563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:52.010929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-1d4e8722-ddcb-4301-ac34-2819e179f818 req-ccf55743-d146-457c-b2b8-23addacc5ee5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:52.011382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-1d4e8722-ddcb-4301-ac34-2819e179f818 req-ccf55743-d146-457c-b2b8-23addacc5ee5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:52.025161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-1d4e8722-ddcb-4301-ac34-2819e179f818 req-ccf55743-d146-457c-b2b8-23addacc5ee5 service nova] Creating event network-vif-deleted:f95dc9f5-1a3c-4215-a5ac-f9399e3cbc25 for instance 7064ed49-237d-441d-a087-70b195245624 on hv-cinder-maste May 23 13:06:52.041220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-1d4e8722-ddcb-4301-ac34-2819e179f818 req-ccf55743-d146-457c-b2b8-23addacc5ee5 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056915 May 23 13:06:52.042463 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 364/728] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:06:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:06:52.194745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:52.212685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ea6e9506-c53c-4ddb-9cf2-52951fd8e59e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.243637 May 23 13:06:52.216601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 365/729] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:51 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:52.641106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-834257113", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "27240b79-2fbf-4ada-bcf2-ef920669591c"}], "security_groups": [{"name": "tempest-securitygroup--1061658672"}], "key_name": "tempest-keypair-1934053805", "user_data": "}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:52.752207 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] validate_networks() for [('27240b79-2fbf-4ada-bcf2-ef920669591c', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:06:52.966745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:52.967279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:52.974035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:52.974668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:52.975275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:52.975702 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:53.008162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:53.008615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:53.009063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lazy-loading 'fault' on Instance uuid 7064ed49-237d-441d-a087-70b195245624 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:06:53.052760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:53.060997 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-92b57862-026a-49c5-8d7e-07785b88f7ad tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 200 len: 1231 microversion: 2.1 time: 0.098089 May 23 13:06:53.061660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 366/730] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:52 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 1231 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:53.232171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:53.233049 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:53.240410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:53.240861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:53.241636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:53.242112 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:53.257225 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:06:53.257785 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:06:53.258448 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:06:53.258783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:06:53.259208 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:06:53.268187 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:53.268709 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:53.269518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:53.269991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:53.295510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:06:53.303141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:06:53.315981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:06:53.318450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:06:53.329577 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:53.383923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:53.392492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a157fc0c-eef9-47d1-bcc1-b2bc2e261fdd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.164812 May 23 13:06:53.393054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 367/731] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:53 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:53.437729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dae4650f-36c0-40bd-b8fc-1eb5126a6edb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.801787 May 23 13:06:53.438405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 365/732] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:06:52 2022] POST /compute/v2.1/servers => generated 402 bytes in 803 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:06:53.448372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-33cae321-88bc-41b9-b7ef-28060ba206d7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:53.449054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-33cae321-88bc-41b9-b7ef-28060ba206d7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:53.504619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-33cae321-88bc-41b9-b7ef-28060ba206d7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:53.511477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-33cae321-88bc-41b9-b7ef-28060ba206d7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1241 microversion: 2.1 time: 0.066831 May 23 13:06:53.512047 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 368/733] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:53 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1241 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:54.082728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:54.083357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:54.091839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.092294 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.093734 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.094204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.107447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] HTTP exception thrown: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Request method failure captured: May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 HTTP/1.1 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WcprMIovHsqBKyUVjs61zB0JHcTlcKlthe0PJsLBmFXx2-NmHfOhkAWRQ2FhhVNCOeow5WG3z8p74dYLMbclpdujgDEiBMpyPbs8Mnd1bWc18nvhNt6DgJNwzZ1GN_L5Jbuv8SD2jrQMOf6_1IzrjEk9G8QFIu7_8buaPgmv53Bs0 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:06:54.109132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-TestVolumeBootPattern-2089392200 May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-TestVolumeBootPattern-2089392200-project May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: c6a6096760734ac49ce2f26c6065b701 May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-TestVolumeBootPattern-2089392200-project May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:06:54.110869 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '7064ed49-237d-441d-a087-70b195245624'} May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:06:54.111974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:06:54.113548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:54.114973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Returning 404 to user: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:06:54.114973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0c87f8b8-1960-4f59-9feb-5bfb3eb427a4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 404 len: 111 microversion: 2.1 time: 0.033474 May 23 13:06:54.114973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 366/734] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:54 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:06:54.159387 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1167634320", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1848200419", "security_groups": [{"name": "tempest-secgroup-smoke-267520630"}], "block_device_mapping_v2": [{"uuid": "5f90287d-4df4-48dc-8bb0-9c8de261c2fd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b8997ea7-0013-4a19-8df5-a24d1eca94b5"}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:54.272193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] validate_networks() for [('b8997ea7-0013-4a19-8df5-a24d1eca94b5', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:06:54.410321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:54.410929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:54.417377 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.417834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.418614 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.419045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.446554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:06:54.606176 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:54.614794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-67cb9f2c-ae54-4aaf-93de-ab437afa209d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.208745 May 23 13:06:54.615364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 367/735] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:54 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:54.619794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:54.620400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:54.627593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.627980 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.628756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.629196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.715211 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:54.722711 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e7abd3bc-d49e-4997-aa25-e75e69627586 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1245 microversion: 2.1 time: 0.106377 May 23 13:06:54.723519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 368/736] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:54 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1245 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:54.746363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:06:54.746363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:06:54.746594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:06:54.746923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:06:54.747327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:06:54.756647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.757083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.757866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:54.758320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:54.774413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:06:54.778245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for user c6a6096760734ac49ce2f26c6065b701 and project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:06:54.837725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:06:54.839934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='5f90287d-4df4-48dc-8bb0-9c8de261c2fd',volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:06:55.295714 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d86e8dc8-4653-478d-8228-aad8b52bd3c4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.142672 May 23 13:06:55.298316 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 369/737] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:06:54 2022] POST /compute/v2.1/servers => generated 405 bytes in 1146 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:06:55.307959 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d0afd14c-2b52-420b-ae2c-f9389019f3f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:55.308574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d0afd14c-2b52-420b-ae2c-f9389019f3f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:55.386014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d0afd14c-2b52-420b-ae2c-f9389019f3f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:55.394658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d0afd14c-2b52-420b-ae2c-f9389019f3f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1095 microversion: 2.1 time: 0.090550 May 23 13:06:55.404071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 369/738] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:55 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1095 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:55.631885 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:55.632511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:55.638721 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:55.641362 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:55.641362 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:55.641362 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:55.739695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:55.740310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:55.746896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:55.747344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:55.748143 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:55.748576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:55.772879 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:55.781282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-758126b0-98a7-4362-bece-d389c3ce4085 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.153165 May 23 13:06:55.791261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 370/739] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:55 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:55.814422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:55.821855 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-83e20140-20c4-49ce-ab64-27488ae1f825 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1293 microversion: 2.1 time: 0.086339 May 23 13:06:55.822443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 370/740] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:55 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1293 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:56.412066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:56.412674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:56.419209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:56.423372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:56.423372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:56.423372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:56.479613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:56.487676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7ba33e07-383b-4a3c-9789-ce5524f548c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1145 microversion: 2.1 time: 0.079445 May 23 13:06:56.488237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 371/741] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:56 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1145 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:56.799562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:56.800166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:56.806566 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:56.807012 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:56.807765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:56.808194 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:56.840119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:56.840863 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:56.847750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:56.848304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:56.849216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:56.849714 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:56.927825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:56.935279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9d3c3367-bba3-4754-a3a6-fe0dc3d7b4b9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1301 microversion: 2.1 time: 0.099135 May 23 13:06:56.936122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 372/742] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:56 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1301 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:56.972593 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:56.980550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-80b4e09d-03e0-4038-8c0e-92cc5a960894 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1736 microversion: 2.1 time: 0.185788 May 23 13:06:56.981095 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 371/743] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:56 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1736 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:57.505681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:57.506299 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:57.516182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:57.516641 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:57.517652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:57.518273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:57.581555 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:57.588937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-88a99162-0f55-4bc6-bc97-208830362f59 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1193 microversion: 2.1 time: 0.087161 May 23 13:06:57.589524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 373/744] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:57 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1193 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:57.952544 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:57.953133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:57.959863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:57.960304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:57.961054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:57.961483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:57.997377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:57.998014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:58.005213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.007123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.007123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.007123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.025702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:58.033995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fa1e8b3f-b923-4acb-a3e6-ad73832adaca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1299 microversion: 2.1 time: 0.085460 May 23 13:06:58.038025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 372/745] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:57 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1299 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:58.335799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:58.349251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5af2dc08-383c-4523-8558-4235e86e8f71 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1716 microversion: 2.1 time: 0.350972 May 23 13:06:58.349251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 374/746] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:57 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1716 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:58.400689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:58.401331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:58.408291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.408747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.409520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.410154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.521099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] HTTP exception thrown: Cannot 'detach_volume' instance 462787a7-152e-4a6a-9c26-740f68c60a77 while it is in vm_state rescued May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Request method failure captured: May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments/c27f6163-8522-414a-aa9b-20eb17ae7934 HTTP/1.1 May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VVZ6qL1baoXQCGJ4rKxwU3PdgDazbVRgPzsvNfj6Z6e7pYM7EagYaFpYssjuDdpTcMmzVLV3_9gBjZFaKnL-a51HATs8ndSjnf_Aso8WExsxhXHAAytDHLZmW36WO0T3MDIwJfdqsUibcr3o-dkGgwCQnPJhHaHKfTcLTweehZiVM May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:06:58.522904 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: b18e5f1b24634096995068e41d94cad6 May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:06:58.524202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Cannot 'detach_volume' instance 462787a7-152e-4a6a-9c26-740f68c60a77 while it is in vm_state rescued May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'server_id': '462787a7-152e-4a6a-9c26-740f68c60a77', 'id': 'c27f6163-8522-414a-aa9b-20eb17ae7934'} May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 in vm_state rescued. Cannot detach_volume while the instance is in this state. May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:06:58.524888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 462787a7-152e-4a6a-9c26-740f68c60a77 while it is in vm_state rescued May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 462787a7-152e-4a6a-9c26-740f68c60a77 while it is in vm_state rescued May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:06:58.526141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Returning 409 to user: Cannot 'detach_volume' instance 462787a7-152e-4a6a-9c26-740f68c60a77 while it is in vm_state rescued {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:06:58.527859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ea4f3d05-0551-4869-9b3e-ae6f266a3a62 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments/c27f6163-8522-414a-aa9b-20eb17ae7934" status: 409 len: 152 microversion: 2.1 time: 0.128706 May 23 13:06:58.527859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 373/747] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:06:58 2022] DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments/c27f6163-8522-414a-aa9b-20eb17ae7934 => generated 152 bytes in 130 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 23 13:06:58.536339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:06:58.541905 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:58.547222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.547796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.548733 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.549543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.606952 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:58.607562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:58.614412 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.614860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.615624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.616054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.708264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:58.716823 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c5e25511-a4f2-4cad-834e-17c1bd905dba tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1211 microversion: 2.1 time: 0.113833 May 23 13:06:58.717671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 374/748] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:58 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1211 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:58.809461 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b7083c3f-4363-4ba1-920a-f567155cde3a tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "POST /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/action" status: 202 len: 0 microversion: 2.1 time: 0.278611 May 23 13:06:58.811429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 375/749] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:06:58 2022] POST /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/action => generated 0 bytes in 281 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:06:58.831732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:58.832712 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:58.838956 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.839409 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:58.840390 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:58.840837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:59.053831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:59.054857 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:59.061354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:59.062747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:59.063855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:59.064395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:59.290736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:59.299281 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-265f25fd-e99b-450e-bf30-20e71dfaae05 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1724 microversion: 2.1 time: 0.473764 May 23 13:06:59.300054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 375/750] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:58 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1724 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:06:59.475974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:06:59.483380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e2997314-bc31-4ff4-bf58-a92dbf3e4ea9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.434254 May 23 13:06:59.484450 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 376/751] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:59 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:06:59.736732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:06:59.737658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:06:59.743875 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:59.744324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:06:59.745093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:06:59.747828 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:00.056277 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:00.069344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e1e5a86c-5365-4b51-8ccd-8a787bdd0cb5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1278 microversion: 2.1 time: 0.336781 May 23 13:07:00.069953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 376/752] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:06:59 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1278 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:00.322867 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:00.323648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:00.329760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-8d397cdb-54fd-480f-b14b-51b9b74cdba6 req-9b736ef1-1ef1-4026-a751-55b55642802e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db38bec4-b1b0-4f6a-97a4-0a303de2790a", "tag": "8c28d9a8-a92e-43b7-8036-2badecfe93cd"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:00.342449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8d397cdb-54fd-480f-b14b-51b9b74cdba6 req-9b736ef1-1ef1-4026-a751-55b55642802e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:00.345520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:00.346074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:00.346922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:00.347410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:00.347787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8d397cdb-54fd-480f-b14b-51b9b74cdba6 req-9b736ef1-1ef1-4026-a751-55b55642802e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:00.375514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-8d397cdb-54fd-480f-b14b-51b9b74cdba6 req-9b736ef1-1ef1-4026-a751-55b55642802e service nova] Creating event network-changed:8c28d9a8-a92e-43b7-8036-2badecfe93cd for instance db38bec4-b1b0-4f6a-97a4-0a303de2790a on hv-cinder-maste May 23 13:07:00.381451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-8d397cdb-54fd-480f-b14b-51b9b74cdba6 req-9b736ef1-1ef1-4026-a751-55b55642802e service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055224 May 23 13:07:00.382635 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 377/753] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:00.500130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:00.500845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:00.520488 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:00.520946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:00.525961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:00.525961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:01.029482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:01.032463 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=100 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:01.038622 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7e8df88b-b9d3-483a-8948-9c0ff2e996bd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.542842 May 23 13:07:01.039305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 378/754] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:00 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:01.088378 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:01.089148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:01.096342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:01.097074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:01.097856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:01.098388 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:01.105687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:01.114126 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-caeea743-fef6-4735-9d05-eb7e0d20124f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1724 microversion: 2.1 time: 0.795141 May 23 13:07:01.123873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 377/755] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:00 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1724 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:01.392248 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:01.401581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-88fc2a3e-5800-4190-b735-59c47b4c46f4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.318079 May 23 13:07:01.402407 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 379/756] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:01 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:02.055619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:02.063472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:02.063472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.063472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.063795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.064235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.131380 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:02.131930 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:02.138355 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.138799 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.139681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.140461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.247046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:02.255211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4b3fb2c4-fd50-4833-9bf9-2f3d1afcc3ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.203840 May 23 13:07:02.256954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 378/757] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:02 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:02.292906 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:02.300965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2572e9bb-4e54-4dcc-bc47-1ccddde132fd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1724 microversion: 2.1 time: 0.173672 May 23 13:07:02.301520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 380/758] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:02 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1724 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:02.419306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:02.419916 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:02.426622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.427068 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.427850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.428285 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.558035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:02.566103 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2d7ef9d2-f778-4ebf-a1e3-406d79b81c2b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.150612 May 23 13:07:02.566653 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 379/759] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:02 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:02.598594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-3e49f0e8-8029-4e01-867f-45e4fd782316 req-64a34138-1f28-407a-999a-81f22dda6a08 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1ded1afb-ad2c-4b62-a675-0b685e5fa30a", "tag": "bfe7ce80-1e0d-4d50-b178-8416be1a709c"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:02.606580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-3e49f0e8-8029-4e01-867f-45e4fd782316 req-64a34138-1f28-407a-999a-81f22dda6a08 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:02.607024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-3e49f0e8-8029-4e01-867f-45e4fd782316 req-64a34138-1f28-407a-999a-81f22dda6a08 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:02.617991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-3e49f0e8-8029-4e01-867f-45e4fd782316 req-64a34138-1f28-407a-999a-81f22dda6a08 service nova] Creating event network-changed:bfe7ce80-1e0d-4d50-b178-8416be1a709c for instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a on hv-cinder-maste May 23 13:07:02.622563 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-3e49f0e8-8029-4e01-867f-45e4fd782316 req-64a34138-1f28-407a-999a-81f22dda6a08 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.027630 May 23 13:07:02.623166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 381/760] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:03.272233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:03.272929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:03.279120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:03.279562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:03.280945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:03.281431 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:03.316048 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:03.320298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:03.320995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:03.328140 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:03.329458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:03.329458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:03.329799 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:03.625176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:03.628518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:03.629071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:03.635947 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bb64036f-944e-4406-bf33-a53f6351cf4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.367722 May 23 13:07:03.638840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 380/761] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:03 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:03.638956 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-00267ec7-7f2e-456a-8c43-63e039293a6d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1724 microversion: 2.1 time: 0.322496 May 23 13:07:03.638956 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 382/762] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:03 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1724 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:03.642422 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:03.643034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:03.649250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:03.649701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:03.650513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:03.651016 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:03.784545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:03.791893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e2ca12a4-dbd4-49cb-8099-35c1a5915c8a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.154255 May 23 13:07:03.792476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 381/763] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:03 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:04.654946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:04.656467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:04.656579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:04.656949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:04.662527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.662985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.663163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.663608 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.663734 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.664147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.664357 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.664794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.765574 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:04.784290 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=12 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:04.863821 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:04.871354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1179df83-d047-407c-abcd-9a5b2f6eb09e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.220203 May 23 13:07:04.872582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:04.872748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 383/764] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:04 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:04.877049 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:04.877855 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:04.880595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8ff13f4e-84b7-40b4-a7e6-ce5980dbafcd tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1730 microversion: 2.1 time: 0.229159 May 23 13:07:04.881233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 382/765] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:04 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:04.895806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.896309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.897677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.898302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.902566 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:04.903052 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:04.911315 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.911997 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:04.913335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:04.914130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:05.029335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cb0618d0-ca77-4caf-81b3-e720f338753c tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.130534 May 23 13:07:05.035420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 383/766] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:07:04 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-interface => generated 285 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:07:05.312306 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:05.320002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-08e251a9-d3bf-4fc1-b3ff-f8302dbecea5 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.447134 May 23 13:07:05.320567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 384/767] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:04 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:05.889045 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:05.890925 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:05.897171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:05.897630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:05.898431 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:05.898869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:05.970613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=8 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:06.035559 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-478ab4d3-a4e8-4426-80eb-ea20bc6ba90c req-03240169-93a9-484b-989d-49e3eec39cdc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "462787a7-152e-4a6a-9c26-740f68c60a77", "tag": "ca8020dd-47c4-42a6-93d6-b0699126db66"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:06.043060 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:06.048301 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-478ab4d3-a4e8-4426-80eb-ea20bc6ba90c req-03240169-93a9-484b-989d-49e3eec39cdc service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:06.048995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-478ab4d3-a4e8-4426-80eb-ea20bc6ba90c req-03240169-93a9-484b-989d-49e3eec39cdc service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:06.051546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7a0e889b-606e-4a1f-8c1b-6b37e3c152ec tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.166790 May 23 13:07:06.052305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 384/768] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:05 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:06.060773 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-478ab4d3-a4e8-4426-80eb-ea20bc6ba90c req-03240169-93a9-484b-989d-49e3eec39cdc service nova] Creating event network-changed:ca8020dd-47c4-42a6-93d6-b0699126db66 for instance 462787a7-152e-4a6a-9c26-740f68c60a77 on hv-cinder-maste May 23 13:07:06.066349 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-478ab4d3-a4e8-4426-80eb-ea20bc6ba90c req-03240169-93a9-484b-989d-49e3eec39cdc service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.103092 May 23 13:07:06.066936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 385/769] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:07:06.336548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:06.337161 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:06.344080 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:06.344540 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:06.345342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:06.345785 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:06.477756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:06.485853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fcd78fb0-1548-4775-894c-20f6b384c004 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.153117 May 23 13:07:06.486650 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 385/770] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:06 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:07.069428 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:07.070202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:07.076577 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:07.077155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:07.078027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:07.078476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:07.359958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:07.367320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0df6c742-7f0e-429c-b7b8-8ad8b7afbae0 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.302652 May 23 13:07:07.368090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 386/771] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:07 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:07.503384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:07.503918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:07.509902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:07.510376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:07.511156 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:07.511591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:07.638794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:07.647125 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9401644c-967f-4733-b176-74676c3a2c5a tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.148063 May 23 13:07:07.647687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 386/772] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:07 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:08.080780 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-8ef591ae-10a7-472a-b363-51591ea91df2 req-82aabc65-cfb3-4cc9-b46a-5536adba435e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "462787a7-152e-4a6a-9c26-740f68c60a77", "tag": "ca8020dd-47c4-42a6-93d6-b0699126db66"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:08.089144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8ef591ae-10a7-472a-b363-51591ea91df2 req-82aabc65-cfb3-4cc9-b46a-5536adba435e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:08.089583 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-8ef591ae-10a7-472a-b363-51591ea91df2 req-82aabc65-cfb3-4cc9-b46a-5536adba435e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:08.100969 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-8ef591ae-10a7-472a-b363-51591ea91df2 req-82aabc65-cfb3-4cc9-b46a-5536adba435e service nova] Creating event network-changed:ca8020dd-47c4-42a6-93d6-b0699126db66 for instance 462787a7-152e-4a6a-9c26-740f68c60a77 on hv-cinder-maste May 23 13:07:08.105634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-8ef591ae-10a7-472a-b363-51591ea91df2 req-82aabc65-cfb3-4cc9-b46a-5536adba435e service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028645 May 23 13:07:08.106232 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 387/773] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:08.384537 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:08.385151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:08.391558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:08.392004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:08.392758 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:08.393189 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:08.598344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:08.605622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7a912c55-68a9-4718-b575-5d35d418e497 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.224987 May 23 13:07:08.606224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 387/774] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:08 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:08.664433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:08.665127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:08.671842 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:08.672292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:08.673067 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:08.673500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:08.884419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:08.893114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9a1d658b-f512-4d5d-a905-a215360a69c7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.232834 May 23 13:07:08.895066 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 388/775] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:08 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:09.119387 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-252cbf8d-26db-47f2-9f47-d54dcb926f5c req-0dd102b6-960d-4ff4-ae44-30bed75300e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db38bec4-b1b0-4f6a-97a4-0a303de2790a", "name": "network-vif-plugged", "status": "completed", "tag": "8c28d9a8-a92e-43b7-8036-2badecfe93cd"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:09.129871 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-252cbf8d-26db-47f2-9f47-d54dcb926f5c req-0dd102b6-960d-4ff4-ae44-30bed75300e7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:09.130486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-252cbf8d-26db-47f2-9f47-d54dcb926f5c req-0dd102b6-960d-4ff4-ae44-30bed75300e7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:09.142837 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-252cbf8d-26db-47f2-9f47-d54dcb926f5c req-0dd102b6-960d-4ff4-ae44-30bed75300e7 service nova] Creating event network-vif-plugged:8c28d9a8-a92e-43b7-8036-2badecfe93cd for instance db38bec4-b1b0-4f6a-97a4-0a303de2790a on hv-cinder-maste May 23 13:07:09.148773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-252cbf8d-26db-47f2-9f47-d54dcb926f5c req-0dd102b6-960d-4ff4-ae44-30bed75300e7 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033441 May 23 13:07:09.149372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 388/776] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:09.622599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:09.623227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:09.630072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:09.630515 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:09.631266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:09.631698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:09.759595 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:09.768002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2a9f4774-a30b-49d9-8d94-5752eb22cd6a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.149231 May 23 13:07:09.768589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 389/777] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:09 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:09.910288 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:09.910981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:09.917498 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:09.917980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:09.918731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:09.919163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:10.050797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:10.058222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9cfeae18-059e-4b6c-b185-4fa9e4fed1fd tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.152162 May 23 13:07:10.058783 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 389/778] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:09 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:10.785624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:10.786192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:10.792700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:10.793144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:10.793926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:10.794366 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:10.920005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:10.927475 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9fe9a610-84e3-4227-a7bb-b59f831d4aab tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1367 microversion: 2.1 time: 0.146532 May 23 13:07:10.928027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 390/779] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:10 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1367 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:11.075524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:11.076057 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:11.083381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:11.083830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:11.084591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:11.085029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:11.212146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:11.219652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-126d8ab5-d34a-4a76-ac3d-da11cfd0ce50 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.148050 May 23 13:07:11.220206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 390/780] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:11 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:11.944743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:11.945405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:11.952006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:11.952006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:11.953144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:11.953576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:12.087361 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:12.095339 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-339aa55d-9846-40ed-bff1-4c4a33c64e13 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1557 microversion: 2.1 time: 0.154441 May 23 13:07:12.095893 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 391/781] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:11 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1557 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:12.111179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:12.111656 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:12.118181 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:12.118629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:12.119379 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:12.119810 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:12.198602 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3dd99c3e-f61a-44cc-a011-ec59d273d225 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.091192 May 23 13:07:12.199196 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 391/782] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:07:12 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-interface => generated 285 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:12.237208 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:12.237825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:12.243874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:12.244329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:12.245107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:12.245546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:12.639881 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:12.647704 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8f0da51a-c2b6-4b69-992c-44832d584bb0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.414934 May 23 13:07:12.648268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 392/783] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:12 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:13.270223 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-dee2c019-ca92-4015-b30e-fca0f0d04a27 req-b2eb2815-a294-4278-b35e-7a53519426c4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db38bec4-b1b0-4f6a-97a4-0a303de2790a", "tag": "8c28d9a8-a92e-43b7-8036-2badecfe93cd"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:13.278482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-dee2c019-ca92-4015-b30e-fca0f0d04a27 req-b2eb2815-a294-4278-b35e-7a53519426c4 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:13.278936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-dee2c019-ca92-4015-b30e-fca0f0d04a27 req-b2eb2815-a294-4278-b35e-7a53519426c4 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:13.290586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-dee2c019-ca92-4015-b30e-fca0f0d04a27 req-b2eb2815-a294-4278-b35e-7a53519426c4 service nova] Creating event network-changed:8c28d9a8-a92e-43b7-8036-2badecfe93cd for instance db38bec4-b1b0-4f6a-97a4-0a303de2790a on hv-cinder-maste May 23 13:07:13.296126 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-dee2c019-ca92-4015-b30e-fca0f0d04a27 req-b2eb2815-a294-4278-b35e-7a53519426c4 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029950 May 23 13:07:13.297192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 392/784] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:13.664495 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:13.665112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:13.672382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:13.672839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:13.673683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:13.674150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:13.810056 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:13.818676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a97ac4a8-c22e-44ee-89fe-9345450247e7 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.158021 May 23 13:07:13.819236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 393/785] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:13 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:14.836183 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:14.836896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:14.843522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:14.847687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:14.847687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:14.847687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:15.037478 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:15.046257 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bb65c093-0c58-4d93-af57-b5de3755c185 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.214053 May 23 13:07:15.047065 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 393/786] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:14 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:16.063478 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:16.064098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:16.070676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:16.071136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:16.071905 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:16.072348 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:16.209033 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:16.216438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e98dbb72-57c4-4bb9-aa7c-712e1b3fb873 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.157121 May 23 13:07:16.216982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 394/787] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:16 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:17.119865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-ad223e3e-cd94-4d1e-ac1f-e6b82e2990bc req-39d0ad50-87e1-4e13-8587-7c1b4cd1fe12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1ded1afb-ad2c-4b62-a675-0b685e5fa30a", "name": "network-vif-plugged", "status": "completed", "tag": "bfe7ce80-1e0d-4d50-b178-8416be1a709c"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:17.129073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-ad223e3e-cd94-4d1e-ac1f-e6b82e2990bc req-39d0ad50-87e1-4e13-8587-7c1b4cd1fe12 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:17.129516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-ad223e3e-cd94-4d1e-ac1f-e6b82e2990bc req-39d0ad50-87e1-4e13-8587-7c1b4cd1fe12 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:17.142495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-ad223e3e-cd94-4d1e-ac1f-e6b82e2990bc req-39d0ad50-87e1-4e13-8587-7c1b4cd1fe12 service nova] Creating event network-vif-plugged:bfe7ce80-1e0d-4d50-b178-8416be1a709c for instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a on hv-cinder-maste May 23 13:07:17.147523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-ad223e3e-cd94-4d1e-ac1f-e6b82e2990bc req-39d0ad50-87e1-4e13-8587-7c1b4cd1fe12 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031063 May 23 13:07:17.148168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 394/788] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:07:17.233710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:17.234336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:17.241047 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:17.241491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:17.242312 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:17.242745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:17.375467 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:17.382985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-bbd02dcf-2e99-442c-82a4-df1f300ab4c0 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.153182 May 23 13:07:17.383527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 395/789] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:17 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:18.401856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:18.402603 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:18.409528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:18.410089 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:18.410851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:18.411289 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:18.542950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:18.550354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7857d050-d452-407c-8317-5d4feabbd0b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1266 microversion: 2.1 time: 0.153455 May 23 13:07:18.550911 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 395/790] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:18 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:19.566545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:19.567072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:19.573754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:19.574222 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:19.574992 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:19.575422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:19.708802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:19.716167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9b381711-0e4c-4d2c-81e1-2950c5876ddb tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1453 microversion: 2.1 time: 0.153621 May 23 13:07:19.716971 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 396/791] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:19 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:19.734034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:19.734678 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:19.741122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:19.741575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:19.742377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:19.742817 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:19.875989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:19.883623 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cfcdd293-5b7c-48cf-8ecb-21e6d15291b4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1453 microversion: 2.1 time: 0.153982 May 23 13:07:19.884182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 396/792] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:07:19 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:07:21.887268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-efc3ac7d-9b84-4779-a868-6e1c084a63a1 req-3b93ace9-8e3b-4cc0-8a01-111bafd3c3e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1ded1afb-ad2c-4b62-a675-0b685e5fa30a", "tag": "bfe7ce80-1e0d-4d50-b178-8416be1a709c"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:21.895761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-efc3ac7d-9b84-4779-a868-6e1c084a63a1 req-3b93ace9-8e3b-4cc0-8a01-111bafd3c3e9 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:21.896204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-efc3ac7d-9b84-4779-a868-6e1c084a63a1 req-3b93ace9-8e3b-4cc0-8a01-111bafd3c3e9 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:21.908104 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-efc3ac7d-9b84-4779-a868-6e1c084a63a1 req-3b93ace9-8e3b-4cc0-8a01-111bafd3c3e9 service nova] Creating event network-changed:bfe7ce80-1e0d-4d50-b178-8416be1a709c for instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a on hv-cinder-maste May 23 13:07:21.912989 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-efc3ac7d-9b84-4779-a868-6e1c084a63a1 req-3b93ace9-8e3b-4cc0-8a01-111bafd3c3e9 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029418 May 23 13:07:21.913742 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 397/793] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:48.821601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:48.822193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:48.824343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:48.824986 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=81 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:48.829755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:48.830232 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:48.830997 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:48.831436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:48.860037 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-92892e8b-6d70-4b6f-a60e-65fd0e2efdf1 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments/e9a2d9e1-1137-4e4e-a1da-d6db1bcbc8a6" status: 200 len: 194 microversion: 2.20 time: 0.043119 May 23 13:07:48.860614 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 397/794] 10.222.0.30 () {60 vars in 1453 bytes} [Mon May 23 13:07:48 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments/e9a2d9e1-1137-4e4e-a1da-d6db1bcbc8a6 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:07:48.923036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=66 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:48.923824 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=99 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:07:48.929257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:48.929841 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:48.935827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:48.936308 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:48.937063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:48.937499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:49.018916 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lazy-loading 'services' on Instance uuid d16a2136-f506-4eb4-8f03-5dadb2a6b515 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:07:49.280315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a7040da5-8e3b-41d9-857c-dd24eb318dbb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments/e9a2d9e1-1137-4e4e-a1da-d6db1bcbc8a6" status: 202 len: 0 microversion: 2.20 time: 0.358672 May 23 13:07:49.280917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 398/795] 10.222.0.30 () {60 vars in 1456 bytes} [Mon May 23 13:07:48 2022] DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments/e9a2d9e1-1137-4e4e-a1da-d6db1bcbc8a6 => generated 0 bytes in 360 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 23 13:07:49.291737 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:49.292705 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:49.299552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:49.299993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:49.300751 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:49.301280 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:49.329505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-23094acb-cf17-4f1f-9040-ca83abfee05e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044114 May 23 13:07:49.330073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 398/796] 10.222.0.30 () {60 vars in 1342 bytes} [Mon May 23 13:07:49 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:07:50.342604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:50.343789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:50.350574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:50.351020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:50.351781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:50.352213 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:50.381142 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-05c31c24-cb93-45a7-86b7-59feb806923d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043274 May 23 13:07:50.381724 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 399/797] 10.222.0.30 () {60 vars in 1342 bytes} [Mon May 23 13:07:50 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:07:51.392439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:51.393356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:51.401345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:51.401802 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:51.402654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:51.403093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:51.431549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5b873dab-97fe-42f5-a4be-15ffeca86900 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.043104 May 23 13:07:51.432106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 399/798] 10.222.0.30 () {60 vars in 1342 bytes} [Mon May 23 13:07:51 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:07:51.490307 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:51.490802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:51.497157 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:51.497610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:51.498474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:51.498931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:51.524283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:07:51.839288 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1082f02a-48ad-41fe-9912-a80b873731a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 204 len: 0 microversion: 2.20 time: 0.352915 May 23 13:07:51.839871 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 400/799] 10.222.0.30 () {60 vars in 1279 bytes} [Mon May 23 13:07:51 2022] DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 23 13:07:51.848017 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:51.848697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:51.854611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:51.855059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:51.855819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:51.856250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:51.989500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:51.997286 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:51.998797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:52.000014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-45a0256d-55b1-436d-8e69-9ab3a823a680 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.155811 May 23 13:07:52.000736 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 400/800] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:07:51 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:07:53.020719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:53.021418 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:53.028120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:53.028576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:53.029341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:53.029783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:53.155765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:53.164579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:53.166115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:53.167272 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-410aba91-5c03-416c-a62b-c6a0db8423b0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.151357 May 23 13:07:53.167885 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 401/801] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:07:53 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:07:54.186025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:54.186722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:54.194667 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:54.195132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:54.195928 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:54.196369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:54.643570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:54.651830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:54.653400 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:54.654660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e4c08927-2069-4f1d-b15f-94ea290e3263 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1723 microversion: 2.20 time: 0.472642 May 23 13:07:54.655212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 401/802] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:07:54 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1723 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:07:55.100654 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-4675d42a-e7c4-4e0a-8646-4e9eda2726e6 req-e9a9ab20-9e00-46f0-837e-0bfe9c4cd75b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a2136-f506-4eb4-8f03-5dadb2a6b515", "name": "network-vif-deleted", "tag": "cee5aa36-f69a-432b-9f71-0cc93809d9a9"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:55.109267 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4675d42a-e7c4-4e0a-8646-4e9eda2726e6 req-e9a9ab20-9e00-46f0-837e-0bfe9c4cd75b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:55.109708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4675d42a-e7c4-4e0a-8646-4e9eda2726e6 req-e9a9ab20-9e00-46f0-837e-0bfe9c4cd75b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:55.121689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-4675d42a-e7c4-4e0a-8646-4e9eda2726e6 req-e9a9ab20-9e00-46f0-837e-0bfe9c4cd75b service nova] Creating event network-vif-deleted:cee5aa36-f69a-432b-9f71-0cc93809d9a9 for instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 on hv-cinder-maste May 23 13:07:55.126600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-4675d42a-e7c4-4e0a-8646-4e9eda2726e6 req-e9a9ab20-9e00-46f0-837e-0bfe9c4cd75b service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030682 May 23 13:07:55.127499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 402/803] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:07:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:07:55.672553 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:55.673157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:55.682186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:55.682186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:55.682476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:55.682861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:55.717130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:55.717583 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:55.718057 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lazy-loading 'fault' on Instance uuid d16a2136-f506-4eb4-8f03-5dadb2a6b515 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:07:55.766318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:55.775003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:55.776488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:07:55.777665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-25983d88-9f4b-43a5-894f-7e4e35115069 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 200 len: 1370 microversion: 2.20 time: 0.108964 May 23 13:07:55.778247 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 402/804] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:07:55 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 1370 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:07:56.797030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:56.797642 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:07:56.804368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:56.804817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:56.805564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:07:56.806022 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:07:56.819040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] HTTP exception thrown: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Request method failure captured: May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 HTTP/1.1 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WxL9Arqy9CJ88sxiUU1WzkGM6LpcxnqseuKE-TVe7S8w8RyNkHgHIjvL1lYkLK5m9PUAwdK4xH_MK7SQup-GZh75PYIGdv1gv9irohgp-YR6Taun_pGQg5-984lJupPA5HhECuSDlDKy6qX8vNz_jyEpvna6TYgladVilrQILHuFE May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Openstack-Nova-Api-Version: 2.20 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:07:56.820638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5b4d791d4aca45429094aa0fea349041"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: f78ab5496c1e4008b83fa3866601c06c May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:07:56.822455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'd16a2136-f506-4eb4-8f03-5dadb2a6b515'} May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:07:56.823424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:07:56.825149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:07:56.826644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Returning 404 to user: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:07:56.826644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ee91c4be-f5eb-43d6-89fe-37859434225f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 404 len: 111 microversion: 2.20 time: 0.029258 May 23 13:07:56.826644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 403/805] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:07:56 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 13:07:56.830662 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-afec2c9e-0e0a-4f8d-83cf-5128cf4de809 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '.version_select at 0x7f9b1c00a820>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:07:56.904754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-afec2c9e-0e0a-4f8d-83cf-5128cf4de809 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-596350624" status: 204 len: 0 microversion: 2.20 time: 0.077771 May 23 13:07:56.905511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 403/806] 10.222.0.30 () {60 vars in 1258 bytes} [Mon May 23 13:07:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-596350624 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 23 13:07:57.665189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-70fd115b-7889-4d53-9a45-43dd12f4f912 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'create', calling method: .version_select at 0x7f9b12ee0430>, body: {"keypair": {"name": "tempest-keypair-1941422353", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNjjw/qJokC0893xSXYg+EUQlLHw1g+zVxE8YNqlxTYwnU3gIs+IdhGbz8iX8WZbXFZwWOImRUvfAAKPtGmBWEwDl5UNrrUu6wKN2S7PPab+N5/pMiFjySiovlggnSr2xg=="}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:07:57.679028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-70fd115b-7889-4d53-9a45-43dd12f4f912 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for project 5b4d791d4aca45429094aa0fea349041. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:07:57.682794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-70fd115b-7889-4d53-9a45-43dd12f4f912 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for user f78ab5496c1e4008b83fa3866601c06c and project 5b4d791d4aca45429094aa0fea349041. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:07:57.765854 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-70fd115b-7889-4d53-9a45-43dd12f4f912 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.105364 May 23 13:07:57.766492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 404/807] 10.222.0.30 () {62 vars in 1199 bytes} [Mon May 23 13:07:57 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 107 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) May 23 13:08:00.132186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-862107142", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "08745ac9-8019-40f9-9db5-aea6cd887576"}], "security_groups": [{"name": "tempest-securitygroup--1432571176"}], "key_name": "tempest-keypair-1941422353", "user_data": "}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:00.304642 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] validate_networks() for [('08745ac9-8019-40f9-9db5-aea6cd887576', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:08:00.685500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:08:00.685953 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:08:00.686430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:08:00.686867 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:08:00.687307 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:08:00.696293 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:00.696740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:00.697539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:00.698003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:00.714969 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for project 5b4d791d4aca45429094aa0fea349041. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:08:00.718846 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Getting quotas for user f78ab5496c1e4008b83fa3866601c06c and project 5b4d791d4aca45429094aa0fea349041. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:08:00.731577 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:08:00.734119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:08:00.883509 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-63237b93-c007-4b65-ad58-f6aa6f592357 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.755758 May 23 13:08:00.884319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 404/808] 10.222.0.30 () {62 vars in 1187 bytes} [Mon May 23 13:08:00 2022] POST /compute/v2.1/servers => generated 402 bytes in 757 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 23 13:08:00.894436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-04faee7f-0daa-4ea2-bc50-9fc296e10442 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:00.895663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-04faee7f-0daa-4ea2-bc50-9fc296e10442 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:00.950738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-04faee7f-0daa-4ea2-bc50-9fc296e10442 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:00.961176 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-04faee7f-0daa-4ea2-bc50-9fc296e10442 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:00.962700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-04faee7f-0daa-4ea2-bc50-9fc296e10442 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:00.963787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-04faee7f-0daa-4ea2-bc50-9fc296e10442 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1281 microversion: 2.20 time: 0.073513 May 23 13:08:00.964329 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 405/809] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:00 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1281 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:01.982341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:01.982953 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:01.990693 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:01.991984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:01.992772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:01.993211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:02.055150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:02.064282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:02.067846 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:02.067846 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fe1b7266-7b59-4850-95eb-b5f07c1f686f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1285 microversion: 2.20 time: 0.089520 May 23 13:08:02.068308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 405/810] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:01 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1285 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:03.086839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:03.087466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:03.094920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:03.095604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:03.096918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:03.096918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:03.186453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:03.195195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:03.196802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:03.198030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1cd7e614-a2ac-4564-80c0-21e16baee801 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1341 microversion: 2.20 time: 0.115818 May 23 13:08:03.198658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 406/811] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:03 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1341 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:04.216405 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:04.217006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:04.224037 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:04.224512 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:04.225658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:04.226120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:04.427523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:04.436364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:04.438005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:04.439115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fca86132-5972-4af3-85bf-d9e3d93cf2df tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.226683 May 23 13:08:04.444508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 406/812] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:04 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:04.704959 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-da69a11e-6134-43fa-9ff8-3c2d28117e85 req-39d7303f-d635-4d6a-a368-19877dadb770 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:04.714630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-da69a11e-6134-43fa-9ff8-3c2d28117e85 req-39d7303f-d635-4d6a-a368-19877dadb770 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:04.715071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-da69a11e-6134-43fa-9ff8-3c2d28117e85 req-39d7303f-d635-4d6a-a368-19877dadb770 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:04.737195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-da69a11e-6134-43fa-9ff8-3c2d28117e85 req-39d7303f-d635-4d6a-a368-19877dadb770 service nova] Creating event network-changed:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:08:04.742992 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-da69a11e-6134-43fa-9ff8-3c2d28117e85 req-39d7303f-d635-4d6a-a368-19877dadb770 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043270 May 23 13:08:04.743594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 407/813] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:05.456964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:05.457573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:05.465187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:05.465646 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:05.466459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:05.466899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:05.778869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:05.786697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:05.802187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:05.802187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e67479f0-6914-43df-a03d-3ab3d5bc5512 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.336292 May 23 13:08:05.802187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 407/814] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:05 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:06.808096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:06.808706 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:06.814852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:06.815297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:06.816053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:06.816482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:06.947660 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:06.955345 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:06.956812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:06.957912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-283b0c86-000e-4140-88dd-fe9291764a3a tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.154522 May 23 13:08:06.958458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 408/815] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:06 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:07.976640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:07.977218 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:07.983399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:07.983851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:07.984611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:07.985049 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:08.112482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:08.121055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:08.122572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:08.123664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3479e6ec-4e1f-4709-bc9c-e95626c87a12 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.151514 May 23 13:08:08.124221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 408/816] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:07 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:09.141646 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:09.142298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:09.149189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:09.149631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:09.150417 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:09.150922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:09.281698 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:09.290040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:09.291537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:09.292635 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a4eea413-7845-47f8-b938-9614554486a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.154918 May 23 13:08:09.293184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 409/817] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:09 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:10.311675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:10.312282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:10.319872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:10.320315 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:10.321131 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:10.321572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:10.448242 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:10.455954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:10.457429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:10.458531 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e740b909-8d46-465e-ad1e-0546cfa00f0e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.150860 May 23 13:08:10.459679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 409/818] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:10 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:11.338135 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-b89122d4-d596-40fb-b59a-ecfe5acdfd63 req-8185941f-74a9-4529-98b2-7e5135d8c13c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "name": "network-vif-plugged", "status": "completed", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:11.342198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:11.347388 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-b89122d4-d596-40fb-b59a-ecfe5acdfd63 req-8185941f-74a9-4529-98b2-7e5135d8c13c service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:11.352323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-b89122d4-d596-40fb-b59a-ecfe5acdfd63 req-8185941f-74a9-4529-98b2-7e5135d8c13c service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:11.359651 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-b89122d4-d596-40fb-b59a-ecfe5acdfd63 req-8185941f-74a9-4529-98b2-7e5135d8c13c service nova] Creating event network-vif-plugged:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:08:11.364633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-b89122d4-d596-40fb-b59a-ecfe5acdfd63 req-8185941f-74a9-4529-98b2-7e5135d8c13c service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031261 May 23 13:08:11.366280 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 410/819] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:11.477163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:11.477765 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:11.479581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:11.480228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:11.485754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:11.486356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:11.487119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:11.487551 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:11.621107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:11.629102 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:11.631933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:11.632346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-43cf4688-1f77-459f-a84e-bd0d2a8704ae tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.159695 May 23 13:08:11.632944 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 410/820] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:11 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:12.501565 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:12.502166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:12.509336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:12.509810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:12.510586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:12.511018 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:12.590524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lazy-loading 'services' on Instance uuid 462787a7-152e-4a6a-9c26-740f68c60a77 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:08:12.650248 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:12.653279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:12.658136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:12.658588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:12.659344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:12.659777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:12.788225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:12.796928 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:12.798091 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-166815f1-6cc4-4923-9e88-940565143a31 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments/c27f6163-8522-414a-aa9b-20eb17ae7934" status: 202 len: 0 microversion: 2.1 time: 0.300779 May 23 13:08:12.802858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 411/821] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:08:12 2022] DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments/c27f6163-8522-414a-aa9b-20eb17ae7934 => generated 0 bytes in 302 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:08:12.803003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:12.803003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-27d2f6ad-eb85-48dd-8643-56eafd3dd3c8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1407 microversion: 2.20 time: 0.154088 May 23 13:08:12.803003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 411/822] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:12 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:08:12.807756 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:12.808761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:12.814897 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:12.815347 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:12.816110 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:12.816545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:12.844990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-248edc90-799d-4aaa-b4da-b6513f163db1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041226 May 23 13:08:12.845528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 412/823] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:08:12 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:13.818846 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:13.819500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:13.826083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:13.826535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:13.827304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:13.827745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:13.857065 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:13.857996 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:13.864599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:13.865620 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:13.866508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:13.866965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:13.897059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b9f0bd43-3319-4802-96c6-ea95dc05db4f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043794 May 23 13:08:13.897891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 413/824] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:08:13 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:08:13.958294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:13.967010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:13.968569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:13.969707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-619be138-5262-4804-aaeb-ce3301bec140 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1600 microversion: 2.20 time: 0.155293 May 23 13:08:13.970297 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 412/825] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:08:13 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1600 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:08:13.987270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:13.987767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:13.994273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:13.994717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:13.995532 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:13.995975 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:14.074716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ad929e1a-dfeb-4f34-90d2-590447d606a4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.091228 May 23 13:08:14.075305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 414/826] 10.222.0.30 () {60 vars in 1315 bytes} [Mon May 23 13:08:13 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-interface => generated 285 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:08:14.908730 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:14.909827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:14.917270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:14.919978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:14.919978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:14.919978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:14.947883 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-359483db-5391-4ea4-bd74-4e18c9a1d3cc tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043077 May 23 13:08:14.948451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 413/827] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:08:14 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:08:15.023300 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:15.023911 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:15.029676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:15.030166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:15.030925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:15.031363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:15.058371 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:08:15.087017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:15.087213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-13dea449-2ff4-4205-826a-b80c9517335c req-6eb56235-a35f-4446-8695-bd2febba3334 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:15.095723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-13dea449-2ff4-4205-826a-b80c9517335c req-6eb56235-a35f-4446-8695-bd2febba3334 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:15.096171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-13dea449-2ff4-4205-826a-b80c9517335c req-6eb56235-a35f-4446-8695-bd2febba3334 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:15.108451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-13dea449-2ff4-4205-826a-b80c9517335c req-6eb56235-a35f-4446-8695-bd2febba3334 service nova] Creating event network-changed:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:08:15.113541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-13dea449-2ff4-4205-826a-b80c9517335c req-6eb56235-a35f-4446-8695-bd2febba3334 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032190 May 23 13:08:15.114769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 414/828] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:08:15.359805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-57bbfd03-8aca-40ae-976b-94b301850bce tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 204 len: 0 microversion: 2.1 time: 0.341315 May 23 13:08:15.360398 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 415/829] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:08:15 2022] DELETE /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:08:15.369501 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:15.370322 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:15.377151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:15.377605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:15.378408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:15.378850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:15.397962 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:15.516682 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:15.524436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d5e7e94e-601b-4284-a37f-6090f1befd82 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1690 microversion: 2.1 time: 0.159620 May 23 13:08:15.525679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 415/830] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:15 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1690 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:16.542832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:16.543526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:16.550703 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:16.551150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:16.551910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:16.552343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:16.682522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:16.690640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fe24510e-dbbf-4709-b27b-f2c3f49cadb1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1690 microversion: 2.1 time: 0.152282 May 23 13:08:16.691204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 416/831] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:16 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1690 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:17.707642 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:17.708256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:17.714723 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:17.715173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:17.715936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:17.716373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:17.842008 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:17.849831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-736cf49f-8ca6-4481-ac56-f4691f19ce24 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1690 microversion: 2.1 time: 0.145628 May 23 13:08:17.850668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 416/832] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:17 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1690 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:18.868650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:18.869383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:18.876002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:18.876461 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:18.877237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:18.877678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:19.006965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:19.014392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-72813167-7123-4857-afb2-d8df8ff704f5 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1690 microversion: 2.1 time: 0.151294 May 23 13:08:19.014946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 417/833] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:18 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1690 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:19.890459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-627f2408-195b-4bb4-81a6-c19c07770d6d req-e5a01a04-2aea-4e64-802d-edaf1cae650a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "462787a7-152e-4a6a-9c26-740f68c60a77", "name": "network-vif-deleted", "tag": "ca8020dd-47c4-42a6-93d6-b0699126db66"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:19.900381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-627f2408-195b-4bb4-81a6-c19c07770d6d req-e5a01a04-2aea-4e64-802d-edaf1cae650a service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:19.901109 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-627f2408-195b-4bb4-81a6-c19c07770d6d req-e5a01a04-2aea-4e64-802d-edaf1cae650a service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:19.912917 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-627f2408-195b-4bb4-81a6-c19c07770d6d req-e5a01a04-2aea-4e64-802d-edaf1cae650a service nova] Creating event network-vif-deleted:ca8020dd-47c4-42a6-93d6-b0699126db66 for instance 462787a7-152e-4a6a-9c26-740f68c60a77 on hv-cinder-maste May 23 13:08:19.918788 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-627f2408-195b-4bb4-81a6-c19c07770d6d req-e5a01a04-2aea-4e64-802d-edaf1cae650a service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040743 May 23 13:08:19.919394 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 417/834] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:20.032132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:20.032850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:20.040028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=62,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:20.041864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:20.049106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:20.050217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:20.050856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:20.123731 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:20.134246 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fa0312b3-ceed-472f-ad48-5acc54d5bb5e tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 200 len: 1330 microversion: 2.1 time: 0.105752 May 23 13:08:20.137934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 418/835] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:20 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 1330 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:21.152093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:21.152809 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 462787a7-152e-4a6a-9c26-740f68c60a77] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:21.159730 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:21.162865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:21.162865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:21.162865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:21.176004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] HTTP exception thrown: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 could not be found. May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Request method failure captured: May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 HTTP/1.1 May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VVZ6qL1baoXQCGJ4rKxwU3PdgDazbVRgPzsvNfj6Z6e7pYM7EagYaFpYssjuDdpTcMmzVLV3_9gBjZFaKnL-a51HATs8ndSjnf_Aso8WExsxhXHAAytDHLZmW36WO0T3MDIwJfdqsUibcr3o-dkGgwCQnPJhHaHKfTcLTweehZiVM May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:08:21.178244 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: b18e5f1b24634096995068e41d94cad6 May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 could not be found. May 23 13:08:21.180240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '462787a7-152e-4a6a-9c26-740f68c60a77'} May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 could not be found. May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:08:21.181072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 could not be found. May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 could not be found. May 23 13:08:21.182281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:21.184111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Returning 404 to user: Instance 462787a7-152e-4a6a-9c26-740f68c60a77 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:08:21.184111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1882f562-a765-4f88-a553-aa6a0094e6f1 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77" status: 404 len: 111 microversion: 2.1 time: 0.032085 May 23 13:08:21.184111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 418/836] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:21 2022] GET /compute/v2.1/servers/462787a7-152e-4a6a-9c26-740f68c60a77 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:08:21.188757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-70cdad69-9089-4cd2-b386-96e1945eea58 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '.version_select at 0x7f9b12ed9d30>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:21.287181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-70cdad69-9089-4cd2-b386-96e1945eea58 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1174189262" status: 202 len: 0 microversion: 2.1 time: 0.102371 May 23 13:08:21.287749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 419/837] 10.222.0.30 () {58 vars in 1220 bytes} [Mon May 23 13:08:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1174189262 => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:08:24.548955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:24.549740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:24.552492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=61,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:24.557241 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:24.557687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:24.558496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:24.558934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:24.585704 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:08:24.921005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a2cc1f14-c136-4367-aca0-34991fcb37b4 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "DELETE /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 204 len: 0 microversion: 2.1 time: 0.376162 May 23 13:08:24.921676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 419/838] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:08:24 2022] DELETE /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:08:24.930267 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:24.930879 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:24.937598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:24.938076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:24.938867 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:24.939304 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:25.072421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:25.080397 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c8d5d648-2049-4766-a5e5-6a988efe11e6 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1507 microversion: 2.1 time: 0.154635 May 23 13:08:25.080954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 420/839] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:24 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1507 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:26.097487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:26.099232 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:26.104618 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:26.105066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:26.106490 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:26.106929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:26.297195 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:26.304697 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-45b905b3-ad26-4e5b-b344-90aa02523f7d tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1507 microversion: 2.1 time: 0.211391 May 23 13:08:26.305250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 420/840] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:26 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1507 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:26.467408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-97afa9bb-38e6-4962-8b86-e521be905972 req-ac44327c-80f6-4b29-a54c-152c2ebecae2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf0c0b3-5059-405a-b847-07ed7c6edee9", "name": "network-vif-deleted", "tag": "65ac147c-5bdb-47fa-adf1-5827d23a6e48"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:26.476170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-97afa9bb-38e6-4962-8b86-e521be905972 req-ac44327c-80f6-4b29-a54c-152c2ebecae2 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:26.476617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-97afa9bb-38e6-4962-8b86-e521be905972 req-ac44327c-80f6-4b29-a54c-152c2ebecae2 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:26.489161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-97afa9bb-38e6-4962-8b86-e521be905972 req-ac44327c-80f6-4b29-a54c-152c2ebecae2 service nova] Creating event network-vif-deleted:65ac147c-5bdb-47fa-adf1-5827d23a6e48 for instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 on hv-cinder-maste May 23 13:08:26.494372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-97afa9bb-38e6-4962-8b86-e521be905972 req-ac44327c-80f6-4b29-a54c-152c2ebecae2 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031901 May 23 13:08:26.494988 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 421/841] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:27.322120 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:27.322635 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:27.329925 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:27.330371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:27.331119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:27.331554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:27.361311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:27.361752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:27.362221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lazy-loading 'fault' on Instance uuid fbf0c0b3-5059-405a-b847-07ed7c6edee9 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:08:27.403003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:27.410305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5c97361f-b1e2-45bb-99a3-42eb6a2738d2 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 200 len: 1311 microversion: 2.1 time: 0.092169 May 23 13:08:27.410856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 421/842] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:27 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 1311 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:28.428108 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:28.428807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: fbf0c0b3-5059-405a-b847-07ed7c6edee9] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:28.435013 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:28.435675 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:28.436450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:28.436894 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:28.449740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] HTTP exception thrown: Instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 could not be found. May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Request method failure captured: May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 HTTP/1.1 May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VVZ6qL1baoXQCGJ4rKxwU3PdgDazbVRgPzsvNfj6Z6e7pYM7EagYaFpYssjuDdpTcMmzVLV3_9gBjZFaKnL-a51HATs8ndSjnf_Aso8WExsxhXHAAytDHLZmW36WO0T3MDIwJfdqsUibcr3o-dkGgwCQnPJhHaHKfTcLTweehZiVM May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:08:28.451876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: b18e5f1b24634096995068e41d94cad6 May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 could not be found. May 23 13:08:28.453283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'fbf0c0b3-5059-405a-b847-07ed7c6edee9'} May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 could not be found. May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:08:28.454166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 could not be found. May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 could not be found. May 23 13:08:28.455599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:28.457082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Returning 404 to user: Instance fbf0c0b3-5059-405a-b847-07ed7c6edee9 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:08:28.457082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-895745ca-f828-4ade-9da6-0e3d4fce8381 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9" status: 404 len: 111 microversion: 2.1 time: 0.029829 May 23 13:08:28.457082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 422/843] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:28 2022] GET /compute/v2.1/servers/fbf0c0b3-5059-405a-b847-07ed7c6edee9 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:08:28.461729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:28.462248 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:28.469504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:28.469968 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:28.470722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:28.471151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:28.495173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:08:28.738114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-99845550-2386-4a1d-90d6-0b5e5b3beff0 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "DELETE /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 204 len: 0 microversion: 2.1 time: 0.279874 May 23 13:08:28.738698 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 422/844] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:08:28 2022] DELETE /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 13:08:28.748355 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:28.748968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:28.754928 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:28.755373 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:28.756174 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:28.756607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:28.887874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:28.896243 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1fa77164-78f9-423e-8a8f-63b1aca59250 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.153179 May 23 13:08:28.896898 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 423/845] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:28 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:29.915098 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:29.915855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:29.923556 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:29.924004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:29.924782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:29.925237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:30.224139 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:30.231503 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e381bbaf-73e2-4e2e-b6c0-6b79286e1eed tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1522 microversion: 2.1 time: 0.320750 May 23 13:08:30.232134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 423/846] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:29 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1522 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:30.431075 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-491065a7-e82c-4aab-9013-b546a6bc3442 req-06bf532b-d499-47a4-81ae-a705b2432635 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d4562fc-a49b-4d0d-b486-e8d2fb59db7c", "name": "network-vif-deleted", "tag": "381aa96b-0ff9-4824-b7af-8578a2b139b2"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:30.439522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-491065a7-e82c-4aab-9013-b546a6bc3442 req-06bf532b-d499-47a4-81ae-a705b2432635 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:30.442689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-491065a7-e82c-4aab-9013-b546a6bc3442 req-06bf532b-d499-47a4-81ae-a705b2432635 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:30.451694 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-491065a7-e82c-4aab-9013-b546a6bc3442 req-06bf532b-d499-47a4-81ae-a705b2432635 service nova] Creating event network-vif-deleted:381aa96b-0ff9-4824-b7af-8578a2b139b2 for instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c on hv-cinder-maste May 23 13:08:30.456472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-491065a7-e82c-4aab-9013-b546a6bc3442 req-06bf532b-d499-47a4-81ae-a705b2432635 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029256 May 23 13:08:30.457379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 424/847] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:08:31.248817 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:31.249435 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:31.255843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:31.256303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:31.257069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:31.257511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:31.290747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:31.291208 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:31.291649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lazy-loading 'fault' on Instance uuid 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:08:31.337841 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b18e5f1b24634096995068e41d94cad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bda5ee012d84d88a3015baa3b80a7d7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:31.340458 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4109f061-4983-43cd-a264-ec0d3af1a336 tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 200 len: 1311 microversion: 2.1 time: 0.096264 May 23 13:08:31.341016 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 424/848] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:31 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 1311 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:31.399771 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:31.400340 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:31.406615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:31.407057 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:31.407811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:31.408239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:31.431404 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-abe6ca7a-6b24-4d34-84a8-d58b597a1b28 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/ips" status: 200 len: 149 microversion: 2.1 time: 0.036176 May 23 13:08:31.438758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 425/849] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:08:31 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/ips => generated 149 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:32.358222 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:32.358779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] [instance: 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:32.367159 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:32.367761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:32.368826 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:32.369428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:32.382936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] HTTP exception thrown: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c could not be found. May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Request method failure captured: May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c HTTP/1.1 May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VVZ6qL1baoXQCGJ4rKxwU3PdgDazbVRgPzsvNfj6Z6e7pYM7EagYaFpYssjuDdpTcMmzVLV3_9gBjZFaKnL-a51HATs8ndSjnf_Aso8WExsxhXHAAytDHLZmW36WO0T3MDIwJfdqsUibcr3o-dkGgwCQnPJhHaHKfTcLTweehZiVM May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:08:32.384532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5bda5ee012d84d88a3015baa3b80a7d7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 5bda5ee012d84d88a3015baa3b80a7d7 May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-553231878 May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: b18e5f1b24634096995068e41d94cad6 May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-ServerRescueNegativeTestJSON-553231878-project May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c could not be found. May 23 13:08:32.386160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '3d4562fc-a49b-4d0d-b486-e8d2fb59db7c'} May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c could not be found. May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:08:32.387085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c could not be found. May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c could not be found. May 23 13:08:32.388526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:32.390524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] Returning 404 to user: Instance 3d4562fc-a49b-4d0d-b486-e8d2fb59db7c could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:08:32.390524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0ef9aa55-9e81-43b2-8aa2-ca1b194d152f tempest-ServerRescueNegativeTestJSON-553231878 tempest-ServerRescueNegativeTestJSON-553231878-project] 10.222.0.30 "GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c" status: 404 len: 111 microversion: 2.1 time: 0.031591 May 23 13:08:32.390524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 425/850] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:32 2022] GET /compute/v2.1/servers/3d4562fc-a49b-4d0d-b486-e8d2fb59db7c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:08:33.072117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b43f9647-38bc-450a-a5e9-459f75bef6e1"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:33.073171 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:33.080346 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:33.081490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:33.082295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:33.082736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:33.837183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-69601027-b422-43c7-8b18-2985faea687f tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.768976 May 23 13:08:33.837920 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 426/851] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:08:33 2022] POST /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments => generated 194 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:41.377686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b43f9647-38bc-450a-a5e9-459f75bef6e1"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:41.378923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:41.385261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:41.385801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:41.386595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:41.387251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:41.468157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Invalid volume: volume b43f9647-38bc-450a-a5e9-459f75bef6e1 is already attached to instances: db38bec4-b1b0-4f6a-97a4-0a303de2790a May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: POST /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments HTTP/1.1 May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 74 May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:08:41.470094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {"volumeAttachment": {"volumeId": "b43f9647-38bc-450a-a5e9-459f75bef6e1"}} May 23 13:08:41.471676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Invalid volume: volume b43f9647-38bc-450a-a5e9-459f75bef6e1 is already attached to instances: db38bec4-b1b0-4f6a-97a4-0a303de2790a May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'server_id': 'db38bec4-b1b0-4f6a-97a4-0a303de2790a', 'body': {'volumeAttachment': {'volumeId': 'b43f9647-38bc-450a-a5e9-459f75bef6e1'}}} May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume b43f9647-38bc-450a-a5e9-459f75bef6e1 is already attached to instances: db38bec4-b1b0-4f6a-97a4-0a303de2790a May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:41.472572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume b43f9647-38bc-450a-a5e9-459f75bef6e1 is already attached to instances: db38bec4-b1b0-4f6a-97a4-0a303de2790a May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:41.473750 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume b43f9647-38bc-450a-a5e9-459f75bef6e1 is already attached to instances: db38bec4-b1b0-4f6a-97a4-0a303de2790a May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 400 to user: Invalid volume: volume b43f9647-38bc-450a-a5e9-459f75bef6e1 is already attached to instances: db38bec4-b1b0-4f6a-97a4-0a303de2790a {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-2a2aeee3-72ab-437f-a22b-7cba209f7d26 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.098271 May 23 13:08:41.475384 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 426/852] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:08:41 2022] POST /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments => generated 174 bytes in 99 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 13:08:41.537185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:41.537795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:41.544106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:41.545477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:41.545477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:41.545723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:41.624093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lazy-loading 'services' on Instance uuid db38bec4-b1b0-4f6a-97a4-0a303de2790a {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:08:41.833533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-aedb9b7d-7959-4e85-b703-9d393f13cf83 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments/b43f9647-38bc-450a-a5e9-459f75bef6e1" status: 202 len: 0 microversion: 2.1 time: 0.300969 May 23 13:08:41.834097 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 427/853] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:08:41 2022] DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments/b43f9647-38bc-450a-a5e9-459f75bef6e1 => generated 0 bytes in 303 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:08:41.842535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:41.843451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:41.850580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:41.851029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:41.851814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:41.852233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:41.882326 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d1635e74-3f05-4a4d-a9fd-720025797b3e tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043728 May 23 13:08:41.882878 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 427/854] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:08:41 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:42.894826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:42.895896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:42.902458 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:42.903035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:42.903817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:42.904261 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:42.934412 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1f3743fc-6c91-4b6b-b591-7669c081e529 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044755 May 23 13:08:42.934996 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 428/855] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:08:42 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:43.946378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:43.947219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:43.953605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:43.954079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:43.954835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:43.956016 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:43.983187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3f44d380-0054-4d6f-ac81-db726df6b9e2 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.040806 May 23 13:08:43.983729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 428/856] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:08:43 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:08:44.043150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:44.043634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:44.050037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:44.050494 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:44.051258 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:44.051700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:44.075826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:08:44.440665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-199d4461-0d15-4f3d-bb22-69a4b96a8e0d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 204 len: 0 microversion: 2.1 time: 0.401126 May 23 13:08:44.441319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 429/857] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:08:44 2022] DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:08:44.452194 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:44.453312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:44.462899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:44.463302 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:44.464053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:44.464489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:44.591977 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:44.599965 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d6cdabb0-eab1-48dd-9b60-38e574c291f9 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1681 microversion: 2.1 time: 0.154376 May 23 13:08:44.600548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 429/858] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:44 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1681 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:45.617633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:45.618359 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:45.624770 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:45.625215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:45.625999 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:45.626433 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:45.630317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:UPDATE=4,nova_cell1:SELECT=85,nova_cell1:INSERT=4 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:45.760535 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:45.768961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e1c6c8a8-32d2-4417-bce2-76604f151a14 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1681 microversion: 2.1 time: 0.155383 May 23 13:08:45.769730 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 430/859] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:45 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:46.786374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:46.786977 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:46.793474 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:46.793947 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:46.794702 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:46.795133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:46.799085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:46.926223 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:46.934507 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-24058987-45f2-4b37-a41a-0fd14f1b3001 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1681 microversion: 2.1 time: 0.152079 May 23 13:08:46.935059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 430/860] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:46 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1681 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:47.952266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:47.952810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:47.960096 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:47.960579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:47.961342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:47.961781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:48.094640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:48.103005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ecf17f54-2684-4bb2-b8e5-04131ed332a3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1681 microversion: 2.1 time: 0.154625 May 23 13:08:48.103575 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 431/861] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:47 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1681 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:49.119803 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:49.120414 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:49.128290 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:49.128753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:49.129674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:49.130143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:49.431234 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:49.438709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-14473af4-64df-4d01-a285-82e7be68b94a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1681 microversion: 2.1 time: 0.322859 May 23 13:08:49.449513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 431/862] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:49 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1681 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:50.137270 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-76219c8b-7c7b-452d-840c-8966c4fd1733 req-6087639b-842c-449f-8592-08eb66c47c22 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db38bec4-b1b0-4f6a-97a4-0a303de2790a", "name": "network-vif-deleted", "tag": "8c28d9a8-a92e-43b7-8036-2badecfe93cd"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:50.146073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-76219c8b-7c7b-452d-840c-8966c4fd1733 req-6087639b-842c-449f-8592-08eb66c47c22 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:50.146503 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-76219c8b-7c7b-452d-840c-8966c4fd1733 req-6087639b-842c-449f-8592-08eb66c47c22 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:50.162235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-76219c8b-7c7b-452d-840c-8966c4fd1733 req-6087639b-842c-449f-8592-08eb66c47c22 service nova] Creating event network-vif-deleted:8c28d9a8-a92e-43b7-8036-2badecfe93cd for instance db38bec4-b1b0-4f6a-97a4-0a303de2790a on hv-cinder-maste May 23 13:08:50.167471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-76219c8b-7c7b-452d-840c-8966c4fd1733 req-6087639b-842c-449f-8592-08eb66c47c22 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034333 May 23 13:08:50.176937 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 432/863] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:50.457495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:50.458913 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:50.466548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:50.467354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:50.468199 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:50.468707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:50.544650 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:50.553202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-463fe021-48d2-495f-b0ac-b8743da97f92 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 200 len: 1326 microversion: 2.1 time: 0.100215 May 23 13:08:50.562017 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 432/864] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:50 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 1326 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:08:51.571719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:51.572440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:51.574374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=36,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.579156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:51.579599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:51.580387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:51.580821 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:51.593598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a HTTP/1.1 May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:08:51.595316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:08:51.597005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'db38bec4-b1b0-4f6a-97a4-0a303de2790a'} May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:08:51.597925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:08:51.599181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:08:51.600328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:08:51.600328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c50fc9dd-fe4f-4339-97d3-48c48036976b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 404 len: 111 microversion: 2.1 time: 0.029814 May 23 13:08:51.600328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 433/865] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:51 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:08:51.606416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-434dda28-16e5-4efe-b916-4c1f88ced970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '.version_select at 0x7f9b1bf7c550>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:51.679305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-434dda28-16e5-4efe-b916-4c1f88ced970 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1934053805" status: 202 len: 0 microversion: 2.1 time: 0.077491 May 23 13:08:51.679891 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 433/866] 10.222.0.30 () {58 vars in 1220 bytes} [Mon May 23 13:08:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1934053805 => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:08:52.625350 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f7d8998e-9fec-4cf5-a4ad-1e15a8c8a038 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: .version_select at 0x7f9b12ddcaf0>, body: {"keypair": {"name": "tempest-keypair-1570862977", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOBuevICrCiWcXzAg3FZQED1M37leJz8J5qKWUWDdH+3sz0WDw3QAfZ2GR0RvTgVOYzxAGBucN6PNc3UVIs++0Lar8rNuDPqflqjSlSB9VsLFfMCWs6kfYYN9YpndLFT5w=="}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:52.640237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-f7d8998e-9fec-4cf5-a4ad-1e15a8c8a038 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:08:52.644902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-f7d8998e-9fec-4cf5-a4ad-1e15a8c8a038 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:08:52.750128 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f7d8998e-9fec-4cf5-a4ad-1e15a8c8a038 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.128560 May 23 13:08:52.750758 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 434/867] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:08:52 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:08:55.256756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-987425441", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "27240b79-2fbf-4ada-bcf2-ef920669591c"}], "security_groups": [{"name": "tempest-securitygroup--1880431397"}], "key_name": "tempest-keypair-1570862977", "user_data": "}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:55.265235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.372638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] validate_networks() for [('27240b79-2fbf-4ada-bcf2-ef920669591c', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:08:55.744590 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:08:55.745154 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:08:55.745749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:08:55.746374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:08:55.746859 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:08:55.756435 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:55.756959 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:55.757853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:55.758396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:55.777296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:08:55.781310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Getting quotas for user 257251830f5148e5b9e435c2c13828e8 and project 0ae347c79fe74082906f0d8eb9dc6677. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:08:55.794047 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:08:55.796312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:08:55.943868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4cd62895-343d-40e1-b73e-5f93d923f7ca tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.693015 May 23 13:08:55.944738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 434/868] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:08:55 2022] POST /compute/v2.1/servers => generated 402 bytes in 694 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:08:55.956370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-46c399d8-894d-4284-a6b3-d72a684c2179 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:55.957113 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-46c399d8-894d-4284-a6b3-d72a684c2179 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:56.014445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-46c399d8-894d-4284-a6b3-d72a684c2179 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:56.023260 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-46c399d8-894d-4284-a6b3-d72a684c2179 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1241 microversion: 2.1 time: 0.072181 May 23 13:08:56.024137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 435/869] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:55 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1241 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:57.041076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:57.042025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:57.049186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:57.052670 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:57.052670 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:57.052670 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:57.118545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:57.125980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f217f3bb-c645-45c0-be98-b792109c9ed7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1245 microversion: 2.1 time: 0.088700 May 23 13:08:57.126611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 435/870] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:57 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1245 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:58.149707 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:58.151000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:58.157781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:58.163365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:58.163365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:58.163365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:58.222383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:58.229987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5c3b8dfb-d256-4da5-b88e-7cf6c547afdc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1311 microversion: 2.1 time: 0.084462 May 23 13:08:58.232005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 436/871] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:58 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1311 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:08:59.247700 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:08:59.248314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:08:59.255672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:59.256049 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:59.257023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:59.257619 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:59.460844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:08:59.468952 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-82a349aa-3f40-4c6f-be9d-79a3f091bfeb tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.224994 May 23 13:08:59.469613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 436/872] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:08:59 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:08:59.738253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-79e7d899-9c10-45cc-bf2e-a2e3f26327f5 req-603151aa-3b10-45cd-9235-030cdf832e9f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c1ae1a05-d9d5-48f8-9394-fb053867218c", "tag": "7b35fa86-306f-4e0a-899e-6bd67c41a574"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:08:59.750672 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-79e7d899-9c10-45cc-bf2e-a2e3f26327f5 req-603151aa-3b10-45cd-9235-030cdf832e9f service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:08:59.750672 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-79e7d899-9c10-45cc-bf2e-a2e3f26327f5 req-603151aa-3b10-45cd-9235-030cdf832e9f service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:08:59.763285 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-79e7d899-9c10-45cc-bf2e-a2e3f26327f5 req-603151aa-3b10-45cd-9235-030cdf832e9f service nova] Creating event network-changed:7b35fa86-306f-4e0a-899e-6bd67c41a574 for instance c1ae1a05-d9d5-48f8-9394-fb053867218c on hv-cinder-maste May 23 13:08:59.770190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-79e7d899-9c10-45cc-bf2e-a2e3f26327f5 req-603151aa-3b10-45cd-9235-030cdf832e9f service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036283 May 23 13:08:59.770412 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 437/873] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:08:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:09:00.490343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:00.490953 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:00.497470 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:00.497962 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:00.498738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:00.499205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:00.636113 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:00.644149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-88075910-5974-4f1e-9215-296c937a3bed tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.157189 May 23 13:09:00.649054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 437/874] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:00 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:01.661524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:01.662248 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:01.668478 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:01.668933 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:01.669769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:01.670243 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:01.802107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:01.809427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-05b9ff62-e214-4f56-a202-78df11da509b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.152695 May 23 13:09:01.810045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 438/875] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:01 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:02.826557 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:02.827299 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:02.834343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:02.834818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:02.835581 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:02.836023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:02.972098 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:02.979794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e0b8380f-8872-4672-95f2-85e90776152b tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.157572 May 23 13:09:02.980463 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 438/876] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:02 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:03.996501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:03.997106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:04.003591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:04.004057 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:04.004851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:04.005280 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:04.416540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:04.424200 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c67dbcf5-1e1c-4691-9138-c4547799d608 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.431572 May 23 13:09:04.424775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 439/877] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:03 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:05.441408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:05.442899 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:05.449611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:05.450102 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:05.450860 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:05.451591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:05.589129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:05.597212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-236617a7-f1e3-45a4-a0bd-fbdd0602273a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.159549 May 23 13:09:05.597932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 439/878] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:05 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:05.638607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-dffe78ae-b05a-43c4-b054-340e6144dbf7 req-22b29dfe-e633-4c4a-8b8b-00bd4d34251d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c1ae1a05-d9d5-48f8-9394-fb053867218c", "name": "network-vif-plugged", "status": "completed", "tag": "7b35fa86-306f-4e0a-899e-6bd67c41a574"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:05.648867 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-dffe78ae-b05a-43c4-b054-340e6144dbf7 req-22b29dfe-e633-4c4a-8b8b-00bd4d34251d service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:05.649392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-dffe78ae-b05a-43c4-b054-340e6144dbf7 req-22b29dfe-e633-4c4a-8b8b-00bd4d34251d service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:05.661192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-dffe78ae-b05a-43c4-b054-340e6144dbf7 req-22b29dfe-e633-4c4a-8b8b-00bd4d34251d service nova] Creating event network-vif-plugged:7b35fa86-306f-4e0a-899e-6bd67c41a574 for instance c1ae1a05-d9d5-48f8-9394-fb053867218c on hv-cinder-maste May 23 13:09:05.666234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-dffe78ae-b05a-43c4-b054-340e6144dbf7 req-22b29dfe-e633-4c4a-8b8b-00bd4d34251d service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031425 May 23 13:09:05.666819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 440/879] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:09:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:09:06.614772 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:06.615494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:06.617411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.618176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.624737 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:06.629448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:06.629448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:06.629448 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:06.759556 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:06.767094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7d182aee-4a27-45ab-a888-00f978431840 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1367 microversion: 2.1 time: 0.156631 May 23 13:09:06.767661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 440/880] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:06 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1367 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:07.784970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:07.785943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:07.787514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:07.792907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:07.793349 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:07.794120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:07.794553 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:07.923277 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:07.931046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b0d2fbd4-6713-4728-81d4-d3bb08f49e77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1556 microversion: 2.1 time: 0.150299 May 23 13:09:07.931602 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 441/881] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:07 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1556 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:07.947667 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:07.948172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:07.954561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:07.955009 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:07.955778 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:07.956212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:08.033236 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-01e0b01d-7e3e-477d-a86f-21b65428b6c8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089353 May 23 13:09:08.033835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 441/882] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:09:07 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:09:09.064896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-a4bb4b1b-90a4-44ca-aa4e-22b8373f9077 req-66907f1a-c7c5-412a-8bab-6d87648bf62e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c1ae1a05-d9d5-48f8-9394-fb053867218c", "tag": "7b35fa86-306f-4e0a-899e-6bd67c41a574"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:09.076234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a4bb4b1b-90a4-44ca-aa4e-22b8373f9077 req-66907f1a-c7c5-412a-8bab-6d87648bf62e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:09.076806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a4bb4b1b-90a4-44ca-aa4e-22b8373f9077 req-66907f1a-c7c5-412a-8bab-6d87648bf62e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:09.089336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-a4bb4b1b-90a4-44ca-aa4e-22b8373f9077 req-66907f1a-c7c5-412a-8bab-6d87648bf62e service nova] Creating event network-changed:7b35fa86-306f-4e0a-899e-6bd67c41a574 for instance c1ae1a05-d9d5-48f8-9394-fb053867218c on hv-cinder-maste May 23 13:09:09.094393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-a4bb4b1b-90a4-44ca-aa4e-22b8373f9077 req-66907f1a-c7c5-412a-8bab-6d87648bf62e service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033368 May 23 13:09:09.095152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 442/883] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:09:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:09:33.989851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.991138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=59 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:34.001820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2049213846", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1848200419", "security_groups": [{"name": "tempest-secgroup-smoke-267520630"}], "block_device_mapping_v2": [{"uuid": "1e0528d9-fba9-48ec-b234-f8bfcefdd6ff", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b8997ea7-0013-4a19-8df5-a24d1eca94b5"}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:34.109059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] validate_networks() for [('b8997ea7-0013-4a19-8df5-a24d1eca94b5', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:09:34.484613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:09:34.485079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:09:34.485579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:09:34.486044 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:09:34.486454 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:09:34.495964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:34.496400 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:34.497395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:34.497829 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:34.520355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:09:34.524877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Getting quotas for user c6a6096760734ac49ce2f26c6065b701 and project c7ef8e9bc68d4e60b8993eac9f160a65. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:09:34.585597 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:09:34.587729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='1e0528d9-fba9-48ec-b234-f8bfcefdd6ff',volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:09:35.007400 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7cdedb0b-da0b-458c-9eb6-27b626ecffd8 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.019012 May 23 13:09:35.008340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 442/884] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:09:33 2022] POST /compute/v2.1/servers => generated 405 bytes in 1020 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:09:35.016743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.017753 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=45 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.021930 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c7291536-f163-4663-a6fd-d03b8662ffd1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:35.023829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c7291536-f163-4663-a6fd-d03b8662ffd1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:35.105598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c7291536-f163-4663-a6fd-d03b8662ffd1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:35.114020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c7291536-f163-4663-a6fd-d03b8662ffd1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1095 microversion: 2.1 time: 0.098994 May 23 13:09:35.115107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 443/885] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:35 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1095 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:36.131487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:36.132098 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:36.139980 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:36.140382 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:36.144497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:36.144497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:36.205203 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:36.212577 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-320846cc-a1e9-42a7-97a3-ff499f996ec4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1137 microversion: 2.1 time: 0.085128 May 23 13:09:36.213136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 443/886] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:36 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1137 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:37.233309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:37.234732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:37.241394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:37.241848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:37.242763 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:37.243206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:37.318619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:37.330932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3a088446-0e6a-4a61-802f-8f0686e5b2ea tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1201 microversion: 2.1 time: 0.102602 May 23 13:09:37.332298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 444/887] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:37 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1201 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:38.352973 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:38.353602 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:38.363356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:38.363813 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:38.364597 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:38.365039 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:38.527655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:38.535434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6ccab312-ef4e-48b9-8fa9-5bccf35da0f6 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1278 microversion: 2.1 time: 0.188856 May 23 13:09:38.536007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 444/888] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:38 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1278 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:38.899819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-dec9606a-9a2b-4ca7-bf83-7f5fb3d476b5 req-00ff7f80-9c3d-4152-a58a-605519f17788 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd2f1c08-5bfb-462a-9253-2c3376d9dbd2", "tag": "e5d22321-630c-4fb7-97e9-d2c4e994d264"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:38.920853 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-dec9606a-9a2b-4ca7-bf83-7f5fb3d476b5 req-00ff7f80-9c3d-4152-a58a-605519f17788 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:38.921566 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-dec9606a-9a2b-4ca7-bf83-7f5fb3d476b5 req-00ff7f80-9c3d-4152-a58a-605519f17788 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:38.947315 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-dec9606a-9a2b-4ca7-bf83-7f5fb3d476b5 req-00ff7f80-9c3d-4152-a58a-605519f17788 service nova] Creating event network-changed:e5d22321-630c-4fb7-97e9-d2c4e994d264 for instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 on hv-cinder-maste May 23 13:09:38.959471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-dec9606a-9a2b-4ca7-bf83-7f5fb3d476b5 req-00ff7f80-9c3d-4152-a58a-605519f17788 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063329 May 23 13:09:38.960099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 445/889] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:09:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:09:39.553354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:39.554782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:39.562584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:39.563030 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:39.563812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:39.564280 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:39.750104 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:39.758012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bb481e40-f487-4ef0-80fb-580e8d31a770 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.208434 May 23 13:09:39.758592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 445/890] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:39 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:40.777137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:40.777785 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:40.783973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:40.784427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:40.785189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:40.785627 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:40.925537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:40.935396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9dd99a70-c862-432c-821a-6f8e07159337 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.163638 May 23 13:09:40.936238 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 446/891] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:40 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:41.953252 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:41.953862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:41.960473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:41.960921 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:41.961684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:41.962149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:42.094680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:42.102571 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a8f4f178-0465-46b9-baa8-c835a843c32c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.153103 May 23 13:09:42.112759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 446/892] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:41 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:43.128584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:43.129299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:43.135801 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:43.136284 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:43.137049 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:43.137631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:43.277567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:43.284837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-02ce6e73-6204-44b9-990c-1cc2ce55b564 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.159771 May 23 13:09:43.285594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 447/893] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:43 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:44.303209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:44.303814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:44.311493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:44.311946 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:44.312729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:44.313166 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:44.446014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:44.453800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fc368977-5d6b-4488-ac48-c8f73d4d88d3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.154360 May 23 13:09:44.454413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 447/894] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:44 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:44.675124 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:44.675628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:44.681857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:44.682334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:44.683119 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:44.683560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:44.709122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7c88f760-27e5-4fef-a54d-e381ebd6f223 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/ips" status: 200 len: 153 microversion: 2.20 time: 0.038932 May 23 13:09:44.709837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 448/895] 10.222.0.30 () {60 vars in 1288 bytes} [Mon May 23 13:09:44 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/ips => generated 153 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:09:45.471206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:45.471822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:45.473634 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.474306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.479627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:45.480076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:45.480838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:45.481276 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:45.778815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:45.786420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ab33b89d-33dd-464a-b2da-4bb0a432135c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.319541 May 23 13:09:45.787001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 448/896] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:45 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:46.802926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:46.803883 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:46.806023 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:46.815471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:46.817751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:46.817751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:46.817751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:46.956317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:46.963856 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0cc1dab1-50d7-4b7a-8a0a-92fc6a742811 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.164799 May 23 13:09:46.964407 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 449/897] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:46 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:47.980683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:47.981205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:47.987818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:47.988269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:47.989066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:47.989504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:48.125179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:48.132805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5df84a92-cc15-451e-b0b8-ace1ec816211 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.156035 May 23 13:09:48.133361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 449/898] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:47 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:48.171794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:48.172362 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:48.178697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:48.179148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:48.179907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:48.180346 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:48.204360 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lazy-loading 'flavor' on Instance uuid e959d5ca-3d5d-466d-b0bb-4d50913b4a28 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:09:48.760429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b11da28c-dd66-4091-9ad5-f377649107de tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/action" status: 202 len: 0 microversion: 2.20 time: 0.592385 May 23 13:09:48.761179 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 450/899] 10.222.0.30 () {62 vars in 1318 bytes} [Mon May 23 13:09:48 2022] POST /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/action => generated 0 bytes in 594 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 13:09:48.769439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:48.773230 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:48.776814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:48.777330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:48.778132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:48.778567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:48.920673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:48.926782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:48.928282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:48.929459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-69b687a1-2916-4f91-9726-91a654ebf810 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.163837 May 23 13:09:48.930029 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 450/900] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:48 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:09:49.151009 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:49.151629 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:49.158637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:49.159089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:49.159994 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:49.160424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:49.293307 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:49.300798 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f87d2399-9575-4bbf-9d7c-689724e075f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.154479 May 23 13:09:49.301572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 451/901] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:49 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:49.948053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:49.948673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:49.957504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:49.957504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:49.957504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:49.957815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:50.087026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:50.095153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:50.096638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:50.097787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-91fcf907-7c0c-4ee0-9461-8784d62ec2b6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.153531 May 23 13:09:50.098354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 451/902] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:49 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:50.317969 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:50.318575 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:50.325100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:50.325549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:50.326814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:50.327247 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:50.461726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:50.469845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f4b20618-9621-4736-8b5f-4350be1a024c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.155831 May 23 13:09:50.470453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 452/903] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:50 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:51.120475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:51.121088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:51.128015 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:51.129260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:51.129260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:51.129669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:51.257661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:51.266332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:51.267801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:51.269075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1ff6b970-972b-47ee-a9ab-eb4a7006e6eb tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.152596 May 23 13:09:51.269626 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 452/904] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:51 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:51.487331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:51.487873 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:51.494384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:51.494840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:51.495611 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:51.496052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:51.632523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:51.635242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-92cbd20c-ae11-4c63-a023-4730a3e0e40d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.152592 May 23 13:09:51.635807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 453/905] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:51 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:51.854904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-79688860-1f3c-4eba-867f-7187cc1f9af9 req-a3e8f2f5-304c-4f98-acfd-3002f73445db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd2f1c08-5bfb-462a-9253-2c3376d9dbd2", "name": "network-vif-plugged", "status": "completed", "tag": "e5d22321-630c-4fb7-97e9-d2c4e994d264"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:51.863471 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-79688860-1f3c-4eba-867f-7187cc1f9af9 req-a3e8f2f5-304c-4f98-acfd-3002f73445db service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:51.868046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-79688860-1f3c-4eba-867f-7187cc1f9af9 req-a3e8f2f5-304c-4f98-acfd-3002f73445db service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:51.874864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-79688860-1f3c-4eba-867f-7187cc1f9af9 req-a3e8f2f5-304c-4f98-acfd-3002f73445db service nova] Creating event network-vif-plugged:e5d22321-630c-4fb7-97e9-d2c4e994d264 for instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 on hv-cinder-maste May 23 13:09:51.879664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-79688860-1f3c-4eba-867f-7187cc1f9af9 req-a3e8f2f5-304c-4f98-acfd-3002f73445db service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028425 May 23 13:09:51.880227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 453/906] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:09:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:09:52.287446 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:52.288195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:52.294218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:52.305425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:52.305425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:52.305425 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:52.425257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:52.433201 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:52.434744 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:52.435903 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c0e9b664-7b99-4b85-bc7f-1f05e340abaa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.152402 May 23 13:09:52.436448 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 454/907] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:52 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:52.652903 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:52.653600 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:52.659890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:52.660331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:52.661091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:52.661528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:52.795607 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:52.803086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f1557001-9535-4547-bd0b-4baa2395cf0d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.155085 May 23 13:09:52.803636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 454/908] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:52 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:53.454309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:53.454918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:53.461585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:53.462070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:53.462823 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:53.463285 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:53.595598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:53.603640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:53.605133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:53.606313 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-02c0e377-0ada-44b4-b02b-277730ef0d51 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.156039 May 23 13:09:53.606874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 455/909] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:53 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:53.820650 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:53.821210 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:53.827624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:53.828079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:53.828875 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:53.829310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:53.957318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:53.964755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4ec360e1-320d-4163-b06c-8b2b4baa8ef1 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1266 microversion: 2.1 time: 0.148101 May 23 13:09:53.965310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 455/910] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:53 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:54.625250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:54.625860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:54.632541 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:54.632987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:54.633749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:54.634214 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:54.764657 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:54.773253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:54.774770 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:54.775964 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3934de3c-344a-4816-bad7-a7bcd67a7482 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.155257 May 23 13:09:54.776542 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 456/911] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:54 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:54.982292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:54.982902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:54.990543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:54.990991 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:54.991746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:54.992181 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:55.123315 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:55.131186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3b7fc4b4-9521-4f03-9839-6070bb59fefc tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1454 microversion: 2.1 time: 0.152795 May 23 13:09:55.131752 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 456/912] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:54 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1454 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:55.149443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:55.150092 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:55.156604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:55.157049 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:55.157815 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:55.158276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:55.287269 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:55.294783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a29007fc-92a8-4ff8-9de8-eba2ec8571e3 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1454 microversion: 2.1 time: 0.149406 May 23 13:09:55.295343 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 457/913] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:09:55 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1454 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:09:55.794135 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:55.795002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:55.802166 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:55.802625 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:55.803403 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:55.803847 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:55.986825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:55.995212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:55.996698 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:55.997900 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-14d01dab-7f38-4fab-8015-2f0d021d7418 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.207758 May 23 13:09:55.998458 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 457/914] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:55 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:57.016501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:57.017094 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:57.023876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:57.024316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:57.025065 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:57.025499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:57.263156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:57.271726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:57.273527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:57.274709 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-83bd486c-12e9-4ac7-b86f-db82e559e03b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.262176 May 23 13:09:57.285903 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 458/915] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:57 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:57.534205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-a88ca0f2-3857-4862-a59c-850badb89026 req-b17ef035-ddcc-46bc-a3e0-11f43728762e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd2f1c08-5bfb-462a-9253-2c3376d9dbd2", "tag": "e5d22321-630c-4fb7-97e9-d2c4e994d264"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:09:57.542906 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-a88ca0f2-3857-4862-a59c-850badb89026 req-b17ef035-ddcc-46bc-a3e0-11f43728762e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:57.543361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-a88ca0f2-3857-4862-a59c-850badb89026 req-b17ef035-ddcc-46bc-a3e0-11f43728762e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:57.554844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-a88ca0f2-3857-4862-a59c-850badb89026 req-b17ef035-ddcc-46bc-a3e0-11f43728762e service nova] Creating event network-changed:e5d22321-630c-4fb7-97e9-d2c4e994d264 for instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 on hv-cinder-maste May 23 13:09:57.559848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-a88ca0f2-3857-4862-a59c-850badb89026 req-b17ef035-ddcc-46bc-a3e0-11f43728762e service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030141 May 23 13:09:57.560895 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 458/916] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:09:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:09:58.295411 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:58.296669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:58.303395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:58.303932 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:58.306748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:58.307772 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:58.328825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.582699 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:58.590578 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:58.592360 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:58.594929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a5958d17-9e11-4283-8b1b-9c045bdb6d8c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.303346 May 23 13:09:58.595649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 459/917] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:58 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:09:59.614325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:09:59.615010 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:09:59.621145 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:59.621535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:59.622318 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:09:59.622747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:09:59.801580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:59.809472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:59.810985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:09:59.812294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1db82243-2c3e-4a22-9d31-4c4c5801553c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.202704 May 23 13:09:59.812841 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 459/918] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:09:59 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:00.830974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:00.837663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:00.837663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:00.838137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:00.838908 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:00.839345 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:01.009738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:01.018382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:01.019850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:01.021002 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e2f9e44c-6570-4a60-9a6e-4e3064f56064 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.194329 May 23 13:10:01.021558 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 460/919] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:00 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:02.044587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:02.045130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:02.052611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:02.053055 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:02.053811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:02.054271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:02.072567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.189337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:02.197143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:02.201732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:02.201732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e03d7547-2f3b-48d4-a41c-e7febc3612c0 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.160028 May 23 13:10:02.201732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 460/920] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:02 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:03.220659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:03.221268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:03.227802 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:03.228257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:03.229186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:03.229448 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:03.357599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:03.366278 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:03.372644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:03.372644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9895453c-9404-40f8-a052-7d0cea6054f3 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.152373 May 23 13:10:03.372644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 461/921] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:03 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:04.387929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:04.388620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:04.391438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=52,nova_api:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.396203 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:04.398014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:04.398014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:04.398273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:04.529067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:04.537782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:04.539304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:04.540465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-759a7518-df63-444e-99f8-292688147647 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.156472 May 23 13:10:04.541013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 461/922] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:04 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:05.557830 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:05.558466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:05.565043 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:05.565588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:05.566279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:05.566694 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:05.704053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:05.712360 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:05.714780 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:05.715070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-65db9b45-4cb3-4d32-b46e-456e3084fb5e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.161123 May 23 13:10:05.715625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 462/923] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:05 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:05.854657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-9834dcd2-ee77-44cc-9e1d-8c45c1f82bdd req-1b398588-8285-4743-ab65-ef5778056698 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "name": "network-vif-plugged", "status": "completed", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:05.864238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-9834dcd2-ee77-44cc-9e1d-8c45c1f82bdd req-1b398588-8285-4743-ab65-ef5778056698 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:05.864689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-9834dcd2-ee77-44cc-9e1d-8c45c1f82bdd req-1b398588-8285-4743-ab65-ef5778056698 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:05.876260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-9834dcd2-ee77-44cc-9e1d-8c45c1f82bdd req-1b398588-8285-4743-ab65-ef5778056698 service nova] Creating event network-vif-plugged:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:10:05.881182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-9834dcd2-ee77-44cc-9e1d-8c45c1f82bdd req-1b398588-8285-4743-ab65-ef5778056698 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030235 May 23 13:10:05.881998 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 462/924] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:10:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:10:06.734494 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:06.735287 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:06.741539 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:06.742020 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:06.742786 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:06.743224 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:06.872326 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:06.881009 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:06.882546 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:06.883690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d3e88a14-ac03-4493-9643-626c2eb409c4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.153762 May 23 13:10:06.884237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 463/925] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:06 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:07.903738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:07.904353 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:07.911553 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:07.912001 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:07.912756 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:07.913266 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:08.042633 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:08.050381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:08.051896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:08.053033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-eb1600be-e0ca-4188-9a96-0c345abe124f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.154181 May 23 13:10:08.053578 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 463/926] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:07 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:09.071321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:09.071954 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:09.074397 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=45 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.079235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:09.079681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:09.080435 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:09.080867 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:09.213582 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:09.221732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:09.223242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:09.224482 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6e03d353-e33b-406c-8229-96157c4bde8d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.157020 May 23 13:10:09.225027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 464/927] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:09 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:10.241460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:10.242091 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:10.248839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:10.252852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:10.252852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:10.252852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:10.380275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:10.388273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:10.390314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:10.391688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-57efd4ae-7426-4bc2-8ae3-967e954c3525 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.153993 May 23 13:10:10.392246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 464/928] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:10 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:10.775800 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:10:10.776247 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:10:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:10:11.410384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:11.411011 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:11.417302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:11.417745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:11.418538 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:11.418968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:11.557853 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:11.565725 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:11.567222 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:11.568366 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d726911e-052f-48e7-858d-87effac93fb6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1746 microversion: 2.20 time: 0.161833 May 23 13:10:11.568907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 465/929] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:11 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1746 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:12.586865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:12.587528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:12.594117 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:12.594572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:12.595351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:12.595793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:12.737295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:12.746133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:12.747644 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:12.748826 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bff5969f-ffab-4a19-96fe-7d6e283b4f86 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.166011 May 23 13:10:12.749979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 465/930] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:12 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:13.769769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:13.770395 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:13.777039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:13.777897 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:13.778674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:13.779106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:13.929149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:13.938131 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:13.939705 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:13.941077 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8d77d9a4-cd96-4e2d-a496-2502011b1097 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.176447 May 23 13:10:13.941817 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 466/931] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:13 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:14.960296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:14.961286 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:14.967985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:14.969797 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:14.970624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:14.971067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:15.119202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:15.128678 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:15.130186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:15.131354 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-63bbda99-007f-4ec7-b85f-89ea9e9e6c74 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.174942 May 23 13:10:15.143989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 466/932] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:14 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:16.151352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:16.152764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:16.160358 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:16.161378 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:16.162332 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:16.162851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:16.364444 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:16.373044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:16.375506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:16.376791 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-73f4b8ef-6803-4b39-9544-1a2abdec2a9e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.229863 May 23 13:10:16.377769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 467/933] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:16 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:17.396213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:17.396822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:17.404501 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:17.405818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:17.406894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:17.407411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:17.554950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:17.569221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:17.569221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:17.571805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3e9fe349-dec4-4308-9f47-da87e26213ef tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.176819 May 23 13:10:17.572831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 467/934] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:17 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:18.590976 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:18.591502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:18.598970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:18.599605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:18.600384 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:18.600977 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:18.748257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:18.756789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:18.758295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:18.759431 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8a5fab5b-c80f-4931-9cd6-bb36a8a9ac76 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.172540 May 23 13:10:18.759975 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 468/935] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:18 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:19.778587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:19.779924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:19.787428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:19.788190 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:19.789040 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:19.789769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:19.959427 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:19.970442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:19.972319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:19.973755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-aff04c13-3ba4-48c9-95d1-52d19a708ec9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.199428 May 23 13:10:19.974480 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 468/936] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:19 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:20.995714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:20.996248 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:21.003812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:21.004488 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:21.005420 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:21.006081 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:21.213581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:21.227470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:21.229219 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:21.230851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-892e8a56-01f7-4a23-87c4-d7c8994d882e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.241852 May 23 13:10:21.231382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 469/937] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:20 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:22.248652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:22.249953 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:22.256438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:22.256881 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:22.257632 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:22.258094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:22.399545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:22.409058 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:22.410584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:22.411746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f2a6f9be-623f-4a3b-8b01-47416f2ebb4b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.167173 May 23 13:10:22.412761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 469/938] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:22 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:23.247792 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1607256369", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:23.298811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] validate_networks() for None {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:10:23.431262 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:23.431869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:23.440601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:23.441056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:23.441832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:23.442487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:23.627488 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:10:23.627907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:10:23.628383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:10:23.628786 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:10:23.629221 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:10:23.637864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:23.638335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:23.639161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:23.640263 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:23.644640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:23.652815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:23.654345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:23.655783 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4bef1609-8cf9-4828-b94d-6340cdcd8803 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1741 microversion: 2.20 time: 0.228237 May 23 13:10:23.656069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 470/939] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:23 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1741 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:23.657804 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Getting quotas for project 0ffdc054740c46f990ff4428f11cf07c. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:10:23.661687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Getting quotas for user b19fc36cb5dc4aa9b684352cea0e6412 and project 0ffdc054740c46f990ff4428f11cf07c. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:10:23.672420 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:10:23.674688 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:10:23.816202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-15953a29-a840-4dc4-80af-0c9897486edd tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.572249 May 23 13:10:23.816860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 470/940] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:10:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 574 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:10:23.826846 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-813791d4-51ef-4cef-96d2-da3ab71cbb37 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:23.827467 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-813791d4-51ef-4cef-96d2-da3ab71cbb37 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:23.884345 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-813791d4-51ef-4cef-96d2-da3ab71cbb37 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:23.891779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-813791d4-51ef-4cef-96d2-da3ab71cbb37 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1215 microversion: 2.1 time: 0.068760 May 23 13:10:23.892341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 471/941] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:23 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1215 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:24.675457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:24.676085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:24.682863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:24.683306 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:24.684123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:24.684553 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:24.861106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:24.869128 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:24.870641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:24.871783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c4a06c8d-8ce7-4b22-9534-c5f4cd23c0e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1723 microversion: 2.20 time: 0.201456 May 23 13:10:24.872332 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 471/942] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:24 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1723 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:24.910238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:24.910835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:24.917339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:24.917786 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:24.918572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:24.919005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:25.023488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:25.031420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c64146bb-11b4-41d5-b02b-78c94b983c28 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1219 microversion: 2.1 time: 0.126310 May 23 13:10:25.031994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 472/943] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:24 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1219 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:25.900376 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:25.900985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:25.909090 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:25.909554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:25.910337 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:25.910774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:26.040788 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:26.048224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:26.048829 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:26.048939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:26.050390 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:26.051547 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e41eba92-b13d-4d71-9e56-901ef67f0f2f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1723 microversion: 2.20 time: 0.155600 May 23 13:10:26.052101 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 472/944] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:25 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:26.055402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:26.055847 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:26.056607 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:26.057043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:26.119706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:26.127130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-35a2c4ae-6c91-4a13-aca6-63900666d8f7 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1267 microversion: 2.1 time: 0.082680 May 23 13:10:26.127699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 473/945] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:26 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1267 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:27.069814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:27.070372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:27.076752 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:27.077839 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:27.078631 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:27.079063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:27.145007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:27.145595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:27.152329 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:27.152775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:27.153534 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:27.154024 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:27.223034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:27.230399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b1d6437c-04d6-481f-8763-39fc2c952f4c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1267 microversion: 2.1 time: 0.089305 May 23 13:10:27.231173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 474/946] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:27 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1267 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:27.234161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:27.242092 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:27.243564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:27.244710 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9fd9e191-145b-46e8-a58e-e1b90b247cbf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1723 microversion: 2.20 time: 0.178885 May 23 13:10:27.245235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 473/947] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:27 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1723 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:27.907635 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-c27d288a-9763-43e5-b2be-68bc1995cead req-527c1e44-3bc5-472d-98ee-e0351db0fc40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "name": "network-vif-unplugged", "status": "completed", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:27.915788 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-c27d288a-9763-43e5-b2be-68bc1995cead req-527c1e44-3bc5-472d-98ee-e0351db0fc40 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:27.919854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-c27d288a-9763-43e5-b2be-68bc1995cead req-527c1e44-3bc5-472d-98ee-e0351db0fc40 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:27.927678 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-c27d288a-9763-43e5-b2be-68bc1995cead req-527c1e44-3bc5-472d-98ee-e0351db0fc40 service nova] Creating event network-vif-unplugged:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:10:27.932640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-c27d288a-9763-43e5-b2be-68bc1995cead req-527c1e44-3bc5-472d-98ee-e0351db0fc40 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030254 May 23 13:10:27.933237 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 475/948] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:10:27 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:10:28.246747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:28.247496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:28.254042 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:28.258716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:28.258716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:28.258716 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:28.268567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:28.270427 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:28.278495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:28.279157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:28.281096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:28.281704 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:28.334138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:28.341591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-efda0e9d-598f-4254-a9d1-677d0e809175 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1273 microversion: 2.1 time: 0.098673 May 23 13:10:28.342239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 474/949] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:28 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1273 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:28.503569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:28.511667 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:28.513147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:28.514869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fe71beb2-4b5b-4b89-9d3e-720953c9a711 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1726 microversion: 2.20 time: 0.253927 May 23 13:10:28.515420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 476/950] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:28 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1726 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:28.538212 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7b1fff27-ec8e-473a-a1fb-28846a203903"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:28.539365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:28.545237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:28.545738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:28.547147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:28.547599 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:28.551407 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=89 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:29.360730 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:29.361404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:29.368308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:29.377258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:29.377258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:29.377258 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:29.632292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9b820659-5b2a-4b42-9122-2bc9a7f454e6 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.098107 May 23 13:10:29.633287 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 475/951] 10.222.0.30 () {62 vars in 1363 bytes} [Mon May 23 13:10:28 2022] POST /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-volume_attachments => generated 188 bytes in 1100 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 13:10:29.760392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:29.760913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:29.770087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:29.772457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:29.772457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:29.772457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:29.825916 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:29.836782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e58186d1-e940-421e-bc47-11b799b9c96c tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.479644 May 23 13:10:29.838171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 477/952] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:29 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:30.036346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-81d4b112-6eea-4a2b-8a8a-876a59b66f8a req-df03ff75-1147-49c3-8954-07d9950c76b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77e0e45d-52e4-413b-a475-23882a9f963b", "tag": "8e16adff-ce2d-4faf-89b8-4f39c24361f9"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:30.053770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-81d4b112-6eea-4a2b-8a8a-876a59b66f8a req-df03ff75-1147-49c3-8954-07d9950c76b1 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.059550 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-81d4b112-6eea-4a2b-8a8a-876a59b66f8a req-df03ff75-1147-49c3-8954-07d9950c76b1 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.070582 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-81d4b112-6eea-4a2b-8a8a-876a59b66f8a req-df03ff75-1147-49c3-8954-07d9950c76b1 service nova] Creating event network-changed:8e16adff-ce2d-4faf-89b8-4f39c24361f9 for instance 77e0e45d-52e4-413b-a475-23882a9f963b on hv-cinder-maste May 23 13:10:30.075618 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-81d4b112-6eea-4a2b-8a8a-876a59b66f8a req-df03ff75-1147-49c3-8954-07d9950c76b1 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043040 May 23 13:10:30.076598 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 478/953] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:10:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:10:30.435868 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-903317e2-7cb0-4f61-8de3-becbdc512182 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-volume_attachments/7b1fff27-ec8e-473a-a1fb-28846a203903" status: 202 len: 0 microversion: 2.20 time: 0.679548 May 23 13:10:30.440031 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 476/954] 10.222.0.30 () {60 vars in 1456 bytes} [Mon May 23 13:10:29 2022] DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-volume_attachments/7b1fff27-ec8e-473a-a1fb-28846a203903 => generated 0 bytes in 681 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 23 13:10:30.495780 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:30.497396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:30.503078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.503527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.504305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.504738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.579610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=100 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:30.787816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fbf25745-14b8-4bda-8fa7-38fe54b30a17 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "POST /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/action" status: 202 len: 0 microversion: 2.20 time: 0.296708 May 23 13:10:30.788544 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 479/955] 10.222.0.30 () {62 vars in 1318 bytes} [Mon May 23 13:10:30 2022] POST /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/action => generated 0 bytes in 299 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 23 13:10:30.796672 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:30.799597 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:30.803922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.804365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.805125 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.805576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.855011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:30.855545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:30.861992 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.862445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.863209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:30.863652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:30.979840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:30.990122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:30.994123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:30.996735 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-bef93116-7ea1-4271-90fa-dc321b96aba7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1674 microversion: 2.20 time: 0.203787 May 23 13:10:30.997922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 477/956] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:30 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1674 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:31.027352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:31.034887 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-54f333bf-df2a-4975-a8eb-37323e9edab3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.183952 May 23 13:10:31.035449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 480/957] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:30 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:32.016379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:32.017127 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:32.023972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:32.026680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:32.026680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:32.026680 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:32.051270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:32.051873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:32.062261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:32.062721 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:32.063512 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:32.063958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:32.211077 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:32.219835 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:32.221339 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:32.222530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-33ec2b70-9272-4ce6-895e-73c69251a807 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1674 microversion: 2.20 time: 0.210936 May 23 13:10:32.223091 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 478/958] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:32 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1674 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:32.238207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:32.245491 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c02da287-1455-4ec7-b30d-f0c67c847b7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.198389 May 23 13:10:32.246070 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 481/959] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:32 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:33.240896 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:33.241537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:33.247958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:33.248405 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:33.250193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:33.250636 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:33.262210 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:33.262822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:33.270063 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:33.270510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:33.271275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:33.271711 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:33.458233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:33.462268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:33.466823 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:33.469257 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:33.469725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-472e5e47-63e3-4709-aea4-b06a9263dbb3 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.211389 May 23 13:10:33.470328 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 482/960] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:33 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:33.470449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-82a49603-17f7-4235-a472-c1c6c08d06bf tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1674 microversion: 2.20 time: 0.233455 May 23 13:10:33.471133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 479/961] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:33 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1674 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:34.486633 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:34.487462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:34.489345 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:34.489762 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:34.490176 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:34.490661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=48 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:34.493338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:34.494252 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:34.495806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:34.496320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:34.497143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:34.497834 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:34.503245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:34.503245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:34.503245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:34.503245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:34.797397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:34.802650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:34.806099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-494da7a3-14c1-474c-b025-87415904c6b6 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.323331 May 23 13:10:34.807027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 483/962] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:34 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:34.812106 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:34.813927 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:34.815085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c83121fd-3327-4b75-ad77-e169e32b0bb8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.329883 May 23 13:10:34.815625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 480/963] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:34 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:35.824456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:35.825991 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:35.834336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:35.835670 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:35.848453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:35.851437 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:35.854403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:35.855196 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:35.856245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:35.857588 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:35.861439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:35.862195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:36.129649 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:36.138870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:36.141105 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:36.142945 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-32f1d4cf-ba1e-42be-aa2c-1fcc31617b31 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.312235 May 23 13:10:36.154164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 481/964] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:35 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:36.177696 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:36.186510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-66a2e203-813d-4363-aa39-a0bc2d15e636 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.365310 May 23 13:10:36.190133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 484/965] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:35 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:37.164876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:37.165665 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:37.173130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:37.173581 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:37.174386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:37.174827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:37.207805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:37.208417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:37.215133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:37.215596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:37.216343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:37.216800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:37.372424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:37.388734 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:37.390895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:37.392052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-79f6b364-0fb2-4556-84c8-35fd8f0464d7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.231133 May 23 13:10:37.392608 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 482/966] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:37 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:37.404192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:37.412306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-afeacfa2-1c39-4877-95b8-3d2c2925df3f tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.208872 May 23 13:10:37.412877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 485/967] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:37 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:38.133506 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-a5e24865-bef8-45d5-bc26-59ae4f686e9d req-16d77804-2289-49c1-890d-443c31ceee2e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77e0e45d-52e4-413b-a475-23882a9f963b", "name": "network-vif-plugged", "status": "completed", "tag": "8e16adff-ce2d-4faf-89b8-4f39c24361f9"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:38.143444 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a5e24865-bef8-45d5-bc26-59ae4f686e9d req-16d77804-2289-49c1-890d-443c31ceee2e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:38.143922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a5e24865-bef8-45d5-bc26-59ae4f686e9d req-16d77804-2289-49c1-890d-443c31ceee2e service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:38.156808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-a5e24865-bef8-45d5-bc26-59ae4f686e9d req-16d77804-2289-49c1-890d-443c31ceee2e service nova] Creating event network-vif-plugged:8e16adff-ce2d-4faf-89b8-4f39c24361f9 for instance 77e0e45d-52e4-413b-a475-23882a9f963b on hv-cinder-maste May 23 13:10:38.164215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-a5e24865-bef8-45d5-bc26-59ae4f686e9d req-16d77804-2289-49c1-890d-443c31ceee2e service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034210 May 23 13:10:38.165126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 483/968] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:10:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:10:38.411182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:38.411838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:38.419758 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:38.420201 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:38.421067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:38.421426 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:38.430084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:38.431073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:38.439957 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:38.440648 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:38.441586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:38.442229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:38.656535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:38.660548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:38.664748 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:38.666281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:38.667447 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c2c36b46-1840-4423-bb6d-49cb6659626d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.260535 May 23 13:10:38.667560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-42af9290-f4f8-4628-8b9b-10873be2ad43 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.242594 May 23 13:10:38.668006 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 486/969] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:38 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:38.668180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 484/970] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:38 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:39.692312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:39.694663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:39.695894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:39.696679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:39.698678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:39.711688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:39.712128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:39.719077 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:39.719077 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:39.719367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:39.719367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:39.721517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:39.722003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:39.921898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:39.931854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:39.933428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:39.934636 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b86257f2-5b73-4805-b375-d0ac2e736def tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.249423 May 23 13:10:39.935190 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 487/971] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:39 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:39.942952 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:39.954505 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-448feeae-3ed1-4b67-ad8f-d3964d9cbd29 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.273737 May 23 13:10:39.955457 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 485/972] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:39 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:40.959556 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:40.960897 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:40.970660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:40.971121 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:40.971535 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:40.972666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:40.973177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:40.973177 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:40.983700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:40.984517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:40.985383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:40.986082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:41.216192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:41.221273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:41.243991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4865ff96-5362-400c-a2fa-f6c0e1ab87ac tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.276438 May 23 13:10:41.244792 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 486/973] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:40 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:41.260939 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:41.261810 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:41.267433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b3b0f6d4-4073-4a28-a4ee-5703c1c52da9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.308421 May 23 13:10:41.267433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 488/974] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:40 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:42.262880 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:42.263514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:42.270815 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:42.271353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:42.272234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:42.273000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:42.277242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=87,nova_cell1:INSERT=5,nova_cell1:UPDATE=8 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:42.284996 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:42.291343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:42.292984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:42.293436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:42.294507 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:42.294946 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:42.488182 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:42.495485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9a0baf93-9eb3-4f44-8649-0185a30e164b tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.236695 May 23 13:10:42.496041 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 487/975] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:42 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:42.499450 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:42.509051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:42.510562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:42.511724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bc5b70c3-cf23-44d8-84ac-5d6aac1869f4 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.230991 May 23 13:10:42.512282 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 489/976] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:42 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:43.512560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:43.513311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:43.520012 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:43.520460 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:43.521239 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:43.521677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:43.533721 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:43.534352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:43.541127 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:43.541589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:43.542395 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:43.542837 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:43.759490 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:43.765808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:43.768377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:43.769868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:43.771042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-338db90a-04b0-431e-900e-701d1a21097f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.242409 May 23 13:10:43.771789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-af330cd6-9d70-4505-aaa0-4759cf420c15 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.262813 May 23 13:10:43.771927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 490/977] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:43 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:43.772990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 488/978] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:43 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:44.789549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:44.791440 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:44.791931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:44.792149 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:44.800234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:44.800575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:44.800776 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:44.800964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:44.801703 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:44.801907 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:44.802272 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:44.802768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:45.010881 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:45.018229 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:45.019327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bf8b5938-5d9e-48cd-8a17-1f2c1ad1c2c1 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.233579 May 23 13:10:45.020299 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 491/979] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:44 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:45.026567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:45.028160 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:45.029386 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-74e87a45-885b-4382-959e-94f7a56fb062 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.243416 May 23 13:10:45.029965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 489/980] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:44 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:46.036872 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:46.037477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:46.043887 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:46.044332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:46.045083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:46.045517 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:46.052597 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:46.053210 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:46.059693 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:46.060145 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:46.060907 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:46.061348 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:46.243939 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:46.247771 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:46.251518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1edfde06-d81e-405c-83c1-b21cef0ab58d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.218771 May 23 13:10:46.253256 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 492/981] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:46 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:46.256233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:46.257717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:46.258892 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4cc77a36-2366-4bae-a829-8245c3c6db0b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.210167 May 23 13:10:46.259427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 490/982] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:46 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:47.270227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:47.270837 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:47.276436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:47.277037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:47.277543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:47.278020 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:47.278782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:47.279215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:47.283450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:47.283902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:47.284652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:47.285080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:47.472270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:47.476951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:47.479688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c8981a53-a3a7-408f-b0a1-760b0b214ee8 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.214105 May 23 13:10:47.480261 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 493/983] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:47 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:47.484872 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:47.486385 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:47.487514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-505848a8-eb5c-4bcb-8f74-036ababa729f tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.215185 May 23 13:10:47.488053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 491/984] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:47 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:48.496486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:48.497096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:48.503704 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:48.504158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:48.504608 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:48.505031 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:48.505213 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:48.505339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:48.511850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:48.512233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:48.512985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:48.513414 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:48.720046 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:48.723413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:48.729068 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:48.730582 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:48.730849 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d983c84a-0951-4dd8-9ae6-bcfba013aaa0 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.238221 May 23 13:10:48.731410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 494/985] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:48 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:48.731719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6e85987e-4023-4942-8432-7b36cd76c4e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.230958 May 23 13:10:48.732260 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 492/986] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:48 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:49.747923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:49.748525 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:49.749459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:49.750101 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:49.755097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:49.755580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:49.756292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:49.756717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:49.756845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:49.757235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:49.758021 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:49.758459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:49.951759 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:49.956078 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:49.959709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:49.960981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:49.962528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:49.963679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6467a69a-1a97-47fa-aa45-899ad81aa14d tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.217956 May 23 13:10:49.963979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a4c3baa0-7994-4be9-9c8e-e5f8358b5274 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.219986 May 23 13:10:49.964697 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 493/987] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:49 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 13:10:49.964853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 495/988] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:49 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:50.981626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:50.982430 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:50.983890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:50.984789 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:50.989330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:50.993401 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:50.993401 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:50.993401 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:50.993710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:50.993868 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:50.994717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:50.995154 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:51.188409 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:51.193024 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:51.196122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e19dc334-5d57-459a-a536-7ce34aa5fe4d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1315 microversion: 2.1 time: 0.218434 May 23 13:10:51.196810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 494/989] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:50 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1315 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:51.200935 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:51.202431 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:51.203572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-eb7517ea-cb5e-4419-8739-6963edfd1ea8 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.224637 May 23 13:10:51.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 496/990] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:50 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:52.214168 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:52.214783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:52.221917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.222337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:52.222949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.223106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:52.224906 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.225537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.231593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.232275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.233224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.233660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.434214 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:52.438050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:52.442485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f4a01276-ee78-4268-8ca4-0b167583ff31 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1465 microversion: 2.1 time: 0.232477 May 23 13:10:52.443059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 495/991] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:52 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1465 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:52.446804 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:52.448356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:52.449593 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-74b947db-fc24-47d8-b5a3-db138a2f6309 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.231677 May 23 13:10:52.450250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 497/992] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:52 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:52.481723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:52.482247 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:52.488930 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.489367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.490164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.490606 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.514918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:10:52.797689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dac6c326-7f1b-4715-b7d1-ce208f651e35 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "DELETE /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 204 len: 0 microversion: 2.1 time: 0.319521 May 23 13:10:52.798321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 496/993] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:10:52 2022] DELETE /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:10:52.807114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:52.810561 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:52.814585 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.815032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.815800 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:52.816238 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:52.943118 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:52.950466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-00a9a005-ea51-45fa-9170-2309d69979b5 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1471 microversion: 2.1 time: 0.147188 May 23 13:10:52.951034 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 498/994] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:52 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1471 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:53.468188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:53.468876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:53.475318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:53.475761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:53.476517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:53.476946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:53.607934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:53.616318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:53.617819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:53.618992 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b1f92356-8df4-4ee1-85e8-67c99797242e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.155079 May 23 13:10:53.619549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 497/995] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:53 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:53.968568 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:53.969346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:53.976835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:53.977287 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:53.978094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:53.978533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:54.108466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:54.116083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e29c6ebf-4bb3-44f2-a366-6a8cb08192ff tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1471 microversion: 2.1 time: 0.151365 May 23 13:10:54.116648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 499/996] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:53 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1471 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:54.141108 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-1aa93b17-4f8e-4499-90a6-e566933ed387 req-a4031b6e-7eba-44c5-a9ef-c4723644fdd3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "name": "network-vif-plugged", "status": "completed", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:54.150269 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-1aa93b17-4f8e-4499-90a6-e566933ed387 req-a4031b6e-7eba-44c5-a9ef-c4723644fdd3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:54.152729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-1aa93b17-4f8e-4499-90a6-e566933ed387 req-a4031b6e-7eba-44c5-a9ef-c4723644fdd3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:54.162324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-1aa93b17-4f8e-4499-90a6-e566933ed387 req-a4031b6e-7eba-44c5-a9ef-c4723644fdd3 service nova] Creating event network-vif-plugged:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:10:54.167375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-1aa93b17-4f8e-4499-90a6-e566933ed387 req-a4031b6e-7eba-44c5-a9ef-c4723644fdd3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091184 May 23 13:10:54.167990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 498/997] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:10:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:10:54.637830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:54.638500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:54.644580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:54.645026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:54.645791 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:54.646250 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:55.133928 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:55.134663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:55.140999 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:55.141483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:55.143014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:55.143668 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:55.150592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:55.160806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:55.162437 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:55.164393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b43335f2-3d76-4568-b02d-13449ac969e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.530332 May 23 13:10:55.176826 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 500/998] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:54 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:55.373464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:55.382147 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-40d28b67-81df-48ae-9d83-5601dbf95c7d tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1471 microversion: 2.1 time: 0.253037 May 23 13:10:55.383026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 499/999] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:55 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1471 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:55.694481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-5c3e889e-45d0-46e4-9554-a04c7a404873 req-530972bc-152c-4206-ab07-ef1657b02cc3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77e0e45d-52e4-413b-a475-23882a9f963b", "name": "network-vif-deleted", "tag": "8e16adff-ce2d-4faf-89b8-4f39c24361f9"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:10:55.703235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-5c3e889e-45d0-46e4-9554-a04c7a404873 req-530972bc-152c-4206-ab07-ef1657b02cc3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:55.703716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-5c3e889e-45d0-46e4-9554-a04c7a404873 req-530972bc-152c-4206-ab07-ef1657b02cc3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:55.716243 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-5c3e889e-45d0-46e4-9554-a04c7a404873 req-530972bc-152c-4206-ab07-ef1657b02cc3 service nova] Creating event network-vif-deleted:8e16adff-ce2d-4faf-89b8-4f39c24361f9 for instance 77e0e45d-52e4-413b-a475-23882a9f963b on hv-cinder-maste May 23 13:10:55.722248 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-5c3e889e-45d0-46e4-9554-a04c7a404873 req-530972bc-152c-4206-ab07-ef1657b02cc3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095460 May 23 13:10:55.723031 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 501/1000] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:10:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:10:56.183607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:56.184219 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:56.190991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:56.191450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:56.192218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:56.192662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:56.325514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:56.333860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:56.335379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:56.336558 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fb3f2c63-a4c5-4be2-8781-abe3bda51848 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.156846 May 23 13:10:56.337107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 500/1001] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:56 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:56.400322 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:56.401025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:56.407771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:56.408225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:56.408995 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:56.409436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:56.472954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:56.480461 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-42be7dd6-f038-4f5a-96e4-8dec68b9a553 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1300 microversion: 2.1 time: 0.084697 May 23 13:10:56.481051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 502/1002] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:56 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1300 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:10:57.354526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:57.355163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:57.362503 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:57.362958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:57.363729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:57.364172 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:57.494885 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:57.496496 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:57.498428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:57.505576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:57.507528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:57.508185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:57.508726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:57.509339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:57.510122 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:57.510248 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1de95988-2218-4365-bae0-20810e0f3e13 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1732 microversion: 2.20 time: 0.159320 May 23 13:10:57.510721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 501/1003] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:57 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1732 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:10:57.544748 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:57.545199 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:57.545707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lazy-loading 'fault' on Instance uuid 77e0e45d-52e4-413b-a475-23882a9f963b {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:10:57.590455 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b19fc36cb5dc4aa9b684352cea0e6412', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ffdc054740c46f990ff4428f11cf07c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:57.598773 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-40cba646-ab13-474f-b7b9-a5f613924022 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 200 len: 1305 microversion: 2.1 time: 0.106131 May 23 13:10:57.599332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 503/1004] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:57 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 1305 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:10:58.529834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:58.530513 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:58.537393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:58.537834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:58.538719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:58.539160 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:58.617002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:10:58.617605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] [instance: 77e0e45d-52e4-413b-a475-23882a9f963b] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:10:58.624211 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:58.624660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:58.625445 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:10:58.626631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:10:58.641069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] HTTP exception thrown: Instance 77e0e45d-52e4-413b-a475-23882a9f963b could not be found. May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Request method failure captured: May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b HTTP/1.1 May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4e8ExB0Vkq9OJ8e7FOIPwlskk9D0uyPOADBLoNz0SR_svEy2fqkACQaQIVhkHK6wtGL2asaOEo9pw1z3Ru1HEvP7nJYKNdCAQ5FCaz8QQV7Y6Na4XUAzvLbevFd8Vkz2GBLNOw1CkmQ3xedWFl0pcMe8MF6GxKX3aXZZfLsf8865cU May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ffdc054740c46f990ff4428f11cf07c May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesNegativeTest-1404590671 May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:10:58.642840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ffdc054740c46f990ff4428f11cf07c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ffdc054740c46f990ff4428f11cf07c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ffdc054740c46f990ff4428f11cf07c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ffdc054740c46f990ff4428f11cf07c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesNegativeTest-1404590671 May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ffdc054740c46f990ff4428f11cf07c May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesNegativeTest-1404590671 May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesNegativeTest-1404590671-project May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: b19fc36cb5dc4aa9b684352cea0e6412 May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesNegativeTest-1404590671-project May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance 77e0e45d-52e4-413b-a475-23882a9f963b could not be found. May 23 13:10:58.644589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '77e0e45d-52e4-413b-a475-23882a9f963b'} May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77e0e45d-52e4-413b-a475-23882a9f963b could not be found. May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:10:58.645479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77e0e45d-52e4-413b-a475-23882a9f963b could not be found. May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77e0e45d-52e4-413b-a475-23882a9f963b could not be found. May 23 13:10:58.646828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:10:58.648477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] Returning 404 to user: Instance 77e0e45d-52e4-413b-a475-23882a9f963b could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:10:58.648477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-23819d3f-37cc-4d88-b5f6-ef3089ef9637 tempest-VolumesNegativeTest-1404590671 tempest-VolumesNegativeTest-1404590671-project] 10.222.0.30 "GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b" status: 404 len: 111 microversion: 2.1 time: 0.031518 May 23 13:10:58.648477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 504/1005] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:10:58 2022] GET /compute/v2.1/servers/77e0e45d-52e4-413b-a475-23882a9f963b => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 13:10:58.680533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:58.684277 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:58.689430 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:58.691052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:10:58.692136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3ce8dfc2-27ef-45e1-b901-97ac02525c68 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1719 microversion: 2.20 time: 0.166252 May 23 13:10:58.692679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 502/1006] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:10:58 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1719 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:11:04.927367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:04.928014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:04.930691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=46 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:04.935500 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:04.935943 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:04.936877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:04.937476 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:04.961591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-326aa858-710b-4987-b392-c496b8bea39d tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/ips" status: 200 len: 148 microversion: 2.1 time: 0.038644 May 23 13:11:04.962176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 505/1007] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:11:04 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/ips => generated 148 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:06.529671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "08ce2cc4-38b3-423d-b9f7-e691661fab42"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:06.531142 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:06.538951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:06.544130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:06.544130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:06.544130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:07.333305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-94b71184-54a8-4969-90f2-bfe90a49ebee tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "POST /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.808048 May 23 13:11:07.334292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 503/1008] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:11:06 2022] POST /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments => generated 194 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:16.136754 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:16.138136 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:16.140291 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=52 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:16.145709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:16.146187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:16.146961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:16.147393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:16.233954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lazy-loading 'services' on Instance uuid c1ae1a05-d9d5-48f8-9394-fb053867218c {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:11:16.426661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ff71038d-33dd-45a9-899a-7d1eb91192af tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments/08ce2cc4-38b3-423d-b9f7-e691661fab42" status: 202 len: 0 microversion: 2.1 time: 0.294384 May 23 13:11:16.428336 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 506/1009] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:11:16 2022] DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments/08ce2cc4-38b3-423d-b9f7-e691661fab42 => generated 0 bytes in 297 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:11:16.438981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:16.441218 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:16.446063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:16.454675 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:16.455548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:16.456928 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:16.457528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:16.493233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cd9608bc-9243-4a8b-be88-26b7827c0a93 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060796 May 23 13:11:16.497627 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 504/1010] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:11:16 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:17.506590 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:17.507609 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:17.516335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:17.517042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:17.517970 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:17.518695 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:17.553906 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1c57019b-ec1c-4a7e-ad05-00ad4c8646f7 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051173 May 23 13:11:17.554468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 507/1011] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:11:17 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:18.566816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:18.568328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:18.575526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:18.576216 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:18.576991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:18.577429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:18.613609 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9ee7ad50-b312-4163-a3b1-d6f8145c0d21 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051653 May 23 13:11:18.619294 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 505/1012] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:11:18 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:11:18.726127 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:18.727474 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:18.736239 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:18.737137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:18.738224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:18.741289 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:18.776409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:11:19.119451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c1ac5b3d-836e-46ae-8905-c226793078d3 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 204 len: 0 microversion: 2.1 time: 0.397183 May 23 13:11:19.120172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 508/1013] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:11:18 2022] DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:11:19.130003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:19.130949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:19.151641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:19.153807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:19.159256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:19.159256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:19.344935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:19.358572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d52b76eb-b215-4e8f-b9a0-7308e6557abd tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1680 microversion: 2.1 time: 0.233923 May 23 13:11:19.359463 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 506/1014] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:19 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1680 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:20.377324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:20.378050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:20.385931 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:20.386432 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:20.387216 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:20.387657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:20.597681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:20.606446 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b4f8b023-fa44-4918-b74d-f016dc239189 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1680 microversion: 2.1 time: 0.233306 May 23 13:11:20.615876 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 509/1015] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:20 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1680 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:21.629396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:21.630045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:21.638876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:21.639516 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:21.640749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:21.642555 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:21.886468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:21.894514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d422a5cb-6c7d-444b-933c-3ab2aff39dfa tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1680 microversion: 2.1 time: 0.269833 May 23 13:11:21.895636 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 507/1016] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:21 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1680 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:22.341057 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-21ad1581-d5c1-4b5b-bd30-51dcb90a2df8 req-fd57bf71-c6a3-4125-98f3-3d6d65ebc9c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c1ae1a05-d9d5-48f8-9394-fb053867218c", "name": "network-vif-deleted", "tag": "7b35fa86-306f-4e0a-899e-6bd67c41a574"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:22.354827 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-21ad1581-d5c1-4b5b-bd30-51dcb90a2df8 req-fd57bf71-c6a3-4125-98f3-3d6d65ebc9c5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:22.355575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-21ad1581-d5c1-4b5b-bd30-51dcb90a2df8 req-fd57bf71-c6a3-4125-98f3-3d6d65ebc9c5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:22.377505 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-21ad1581-d5c1-4b5b-bd30-51dcb90a2df8 req-fd57bf71-c6a3-4125-98f3-3d6d65ebc9c5 service nova] Creating event network-vif-deleted:7b35fa86-306f-4e0a-899e-6bd67c41a574 for instance c1ae1a05-d9d5-48f8-9394-fb053867218c on hv-cinder-maste May 23 13:11:22.384679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-21ad1581-d5c1-4b5b-bd30-51dcb90a2df8 req-fd57bf71-c6a3-4125-98f3-3d6d65ebc9c5 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048754 May 23 13:11:22.386035 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 510/1017] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:11:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:22.916888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:22.918713 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:22.932274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:22.933014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:22.934278 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:22.935175 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:22.984532 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:22.986083 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:22.987043 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lazy-loading 'fault' on Instance uuid c1ae1a05-d9d5-48f8-9394-fb053867218c {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:11:23.043064 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '257251830f5148e5b9e435c2c13828e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ae347c79fe74082906f0d8eb9dc6677', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:23.058015 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b9474164-36d0-4144-ba78-7f4048da1961 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 200 len: 1331 microversion: 2.1 time: 0.147642 May 23 13:11:23.059297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 508/1018] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:22 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 1331 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:24.081368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:24.082179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:24.092124 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:24.095825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:24.095825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:24.095825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:24.112664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c HTTP/1.1 May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:24.114388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:24.115793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'c1ae1a05-d9d5-48f8-9394-fb053867218c'} May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:24.116612 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:24.118710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:24.120768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:24.120768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-18e8f533-013d-417c-8cdd-844233e01b77 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 404 len: 111 microversion: 2.1 time: 0.043514 May 23 13:11:24.120768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 511/1019] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:24 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:11:24.128491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-1ffd6cd1-0296-4ca7-90cf-e4461c5adf68 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '.version_select at 0x7f9b12fba0d0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:24.129196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR oslo.messaging._drivers.impl_rabbit [None req-1ffd6cd1-0296-4ca7-90cf-e4461c5adf68 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [819dbb27-e0ee-4967-9b27-a1c2e4337b56] AMQP server on 10.222.0.30:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 13:11:25.137490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [None req-1ffd6cd1-0296-4ca7-90cf-e4461c5adf68 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [819dbb27-e0ee-4967-9b27-a1c2e4337b56] Reconnected to AMQP server on 10.222.0.30:5672 via [amqp] client with port 44080. May 23 13:11:25.198407 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-1ffd6cd1-0296-4ca7-90cf-e4461c5adf68 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1570862977" status: 202 len: 0 microversion: 2.1 time: 1.076980 May 23 13:11:25.199551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 509/1020] 10.222.0.30 () {58 vars in 1220 bytes} [Mon May 23 13:11:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1570862977 => generated 0 bytes in 1079 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:11:27.606197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:27.606764 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:27.615443 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:27.615976 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:27.616876 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:27.617437 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:27.636339 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c HTTP/1.1 May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:27.638115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.639718 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'c1ae1a05-d9d5-48f8-9394-fb053867218c'} May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:27.640706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.643033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:27.644743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:27.644743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c4f0979a-7eb8-401e-a544-9fe6c6319ebc tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 404 len: 111 microversion: 2.1 time: 0.038123 May 23 13:11:27.644743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 512/1021] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:11:27 2022] DELETE /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:11:27.649628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:27.650328 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: c1ae1a05-d9d5-48f8-9394-fb053867218c] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:27.657372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:27.657827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:27.658629 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:27.659074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:27.673474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c HTTP/1.1 May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:11:27.675316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.677375 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'c1ae1a05-d9d5-48f8-9394-fb053867218c'} May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:27.678190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. May 23 13:11:27.679533 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:11:27.680624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance c1ae1a05-d9d5-48f8-9394-fb053867218c could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:27.680624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2b0f829a-1e41-4fc9-8444-88db8e447eb6 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c" status: 404 len: 111 microversion: 2.1 time: 0.032328 May 23 13:11:27.680624 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 510/1022] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:27 2022] GET /compute/v2.1/servers/c1ae1a05-d9d5-48f8-9394-fb053867218c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:11:28.497479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7c611d35-ccd9-4022-952f-d2c5b89b63f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Action: 'create', calling method: .version_select at 0x7f9b1c0eaca0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1658194789", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGE/HN1XQVtBCkqo55stjGeeECWTsCuYEa7Qar+Big2nE/xwN8AQn0hpA5wk8BhA033O2/7KrOtTFXlGxcB8go7ZwCuaGLyDC1Q6DcsFcu0CJJbPwQi027YOoQdV0TMUwA=="}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:28.511458 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-7c611d35-ccd9-4022-952f-d2c5b89b63f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Getting quotas for project 1f2fed134f274a6ebe5ffe183205e269. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:11:28.515413 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-7c611d35-ccd9-4022-952f-d2c5b89b63f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Getting quotas for user 20d9973f32f847a6a54a60405beabdbf and project 1f2fed134f274a6ebe5ffe183205e269. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:11:28.586294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7c611d35-ccd9-4022-952f-d2c5b89b63f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.092661 May 23 13:11:28.586889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 513/1023] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:11:28 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:28.635639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-346675578", "imageRef": "fd1083b4-7e86-4fa1-9efa-8350bd4a6cf6", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1658194789", "networks": [{"uuid": "c41eeeda-f50c-49d1-b23f-c49093f8c20d"}]}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:28.689723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] validate_networks() for [('c41eeeda-f50c-49d1-b23f-c49093f8c20d', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:11:28.946213 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:28.946734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:28.953545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:28.954036 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:28.954805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:28.955235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:28.968376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a HTTP/1.1 May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:28.970014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:28.971669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'db38bec4-b1b0-4f6a-97a4-0a303de2790a'} May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:28.972485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:28.973775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:28.975475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:28.975475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-08995874-7e4b-48d6-a655-428439d4ca55 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 404 len: 111 microversion: 2.1 time: 0.029531 May 23 13:11:28.975475 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 514/1024] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:11:28 2022] DELETE /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 13:11:28.980277 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:28.980936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: db38bec4-b1b0-4f6a-97a4-0a303de2790a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:28.987683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:28.988137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:28.988908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:28.989351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.004314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a HTTP/1.1 May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:29.005855 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:29.007483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'db38bec4-b1b0-4f6a-97a4-0a303de2790a'} May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:29.008334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. May 23 13:11:29.009604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.010832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance db38bec4-b1b0-4f6a-97a4-0a303de2790a could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:29.010832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-19133e76-78f3-43a6-b091-4c0f893693ae tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a" status: 404 len: 111 microversion: 2.1 time: 0.031340 May 23 13:11:29.010832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 515/1025] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:28 2022] GET /compute/v2.1/servers/db38bec4-b1b0-4f6a-97a4-0a303de2790a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 13:11:29.015780 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:29.016281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:29.022749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:29.023206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.023982 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:29.024428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.038081 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 HTTP/1.1 May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:29.039629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.041351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'adac7929-c83b-4f7b-9b97-1b7a48da67a2'} May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:29.042358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.043839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.045359 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:29.045359 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d2b63254-3738-418b-9903-fcbce8b99a72 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 404 len: 111 microversion: 2.1 time: 0.028928 May 23 13:11:29.045359 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 516/1026] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:11:29 2022] DELETE /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:11:29.054123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:29.054816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: adac7929-c83b-4f7b-9b97-1b7a48da67a2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:29.060746 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:29.061192 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.062051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:29.062488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.067647 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:11:29.068152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:11:29.068626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:11:29.069035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:11:29.069443 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:11:29.076629 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 HTTP/1.1 May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:29.078172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'adac7929-c83b-4f7b-9b97-1b7a48da67a2'} May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:29.081459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. May 23 13:11:29.082844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:29.085036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:29.085036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.085351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance adac7929-c83b-4f7b-9b97-1b7a48da67a2 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:29.085519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:29.085668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0d1ce695-a19d-42b2-8d5c-7af31da39133 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2" status: 404 len: 111 microversion: 2.1 time: 0.030111 May 23 13:11:29.085819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:29.086548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 517/1027] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:29 2022] GET /compute/v2.1/servers/adac7929-c83b-4f7b-9b97-1b7a48da67a2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 13:11:29.098638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Getting quotas for project 1f2fed134f274a6ebe5ffe183205e269. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:11:29.103045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Getting quotas for user 20d9973f32f847a6a54a60405beabdbf and project 1f2fed134f274a6ebe5ffe183205e269. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:11:29.113902 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:11:29.116109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fd1083b4-7e86-4fa1-9efa-8350bd4a6cf6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:11:29.283947 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-65fb1fbb-827f-44a0-bb33-1962459614db tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.652075 May 23 13:11:29.284507 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 511/1028] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:11:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 653 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:11:29.294560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-daf381a8-6694-46ed-b3f6-224c366015b6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:29.295169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-daf381a8-6694-46ed-b3f6-224c366015b6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:29.357512 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-daf381a8-6694-46ed-b3f6-224c366015b6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:29.364550 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-daf381a8-6694-46ed-b3f6-224c366015b6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1258 microversion: 2.1 time: 0.073981 May 23 13:11:29.365113 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 518/1029] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:29 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1258 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:30.381707 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e979d642-332a-4ada-a37b-0f6cebf0b65e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:30.382344 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e979d642-332a-4ada-a37b-0f6cebf0b65e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:30.414572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:30.415081 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:30.422840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:30.423393 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:30.424172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:30.424739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:30.436220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e979d642-332a-4ada-a37b-0f6cebf0b65e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:30.438680 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 HTTP/1.1 May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:30.440373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.442096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'e01afbdf-8ced-4f91-b14c-b87b2fcf1506'} May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:30.442949 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.444255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.445799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:30.445799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a9feec4f-e810-41c7-83d3-782eb09aacd8 tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 404 len: 111 microversion: 2.1 time: 0.031256 May 23 13:11:30.445799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 519/1030] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:11:30 2022] DELETE /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:11:30.451071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e979d642-332a-4ada-a37b-0f6cebf0b65e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1258 microversion: 2.1 time: 0.067053 May 23 13:11:30.451071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 512/1031] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:30 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1258 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:30.451245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:30.451245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] [instance: e01afbdf-8ced-4f91-b14c-b87b2fcf1506] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:30.457832 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:30.458326 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:30.459207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:30.459655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:30.473514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] HTTP exception thrown: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Request method failure captured: May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 HTTP/1.1 May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4VUr7RflvwHtJtLUoKek4qGoohIDcCUEqnPFUWCOUWIaxNLOtptzzVQ7Q6T94cu8WvEnb1W1ShpzrXm0iOcOdDp5HHZyYbGIjJ292JPpUDQ0tC9-tJ7JnjilkKAMfmY2aoLS-s-c8chpIqPgMNRbPpJ7iOk3JcnxQYwt0WP2NI0yeM May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:11:30.475128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/0ae347c79fe74082906f0d8eb9dc6677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 0ae347c79fe74082906f0d8eb9dc6677 May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1174665204 May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 257251830f5148e5b9e435c2c13828e8 May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeNegativeTest-1174665204-project May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.476794 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'e01afbdf-8ced-4f91-b14c-b87b2fcf1506'} May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:11:30.477586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. May 23 13:11:30.478922 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:11:30.480346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] Returning 404 to user: Instance e01afbdf-8ced-4f91-b14c-b87b2fcf1506 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:11:30.480346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a4afa4eb-b3f5-41fe-b24b-586001426d4a tempest-AttachVolumeNegativeTest-1174665204 tempest-AttachVolumeNegativeTest-1174665204-project] 10.222.0.30 "GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506" status: 404 len: 111 microversion: 2.1 time: 0.029760 May 23 13:11:30.480346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 520/1032] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:30 2022] GET /compute/v2.1/servers/e01afbdf-8ced-4f91-b14c-b87b2fcf1506 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:11:31.467496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:31.468272 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:31.475723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:31.476167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:31.476941 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:31.477371 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:31.542913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:31.551306 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d6caa48e-02f0-4347-8ce4-b5265df5b2c7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1310 microversion: 2.1 time: 0.087880 May 23 13:11:31.551892 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 513/1033] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:31 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1310 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:32.569317 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:32.570085 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:32.589573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:32.590061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:32.590857 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:32.591304 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:33.093212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:33.104460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7ec3b03d-4725-4fc9-a42c-8a46d10f6b14 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.538866 May 23 13:11:33.106546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 521/1034] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:32 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:34.128350 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:34.129022 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:34.135903 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:34.136579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:34.137415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:34.137865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:34.337390 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:34.361051 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-bd408ea1-c0f1-4885-b7a7-fff201c685ec tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.239744 May 23 13:11:34.366616 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 514/1035] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:34 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:35.383834 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:35.384545 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:35.391993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:35.392536 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:35.393549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:35.394231 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:35.557415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:35.566729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-992c227a-a046-406f-a146-dde49bbd30d4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.186747 May 23 13:11:35.567313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 522/1036] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:35 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:36.583678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:36.584287 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:36.592530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:36.592981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:36.593834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:36.594441 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:37.018203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:37.020367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:37.026984 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f9c0ee46-2e1e-430a-b93d-40d38dd147df tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.447082 May 23 13:11:37.032033 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 515/1037] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:36 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:38.045270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:38.047155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:38.056776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:38.057465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:38.058695 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:38.059365 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:38.216290 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:38.224768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a39b5d28-e0dc-40b2-9ee2-d690acc61294 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.183877 May 23 13:11:38.234749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 523/1038] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:38 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:39.242231 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:39.242842 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:39.245402 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:39.251292 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:39.251742 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:39.252503 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:39.252935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:39.386809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:39.395299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e6cb47a5-3580-4ff9-a025-bcc670fb389e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.157097 May 23 13:11:39.395897 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 516/1039] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:39 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:40.412683 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:40.413869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:40.418101 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:40.429176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:40.429659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:40.430483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:40.430955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:40.571961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:40.579850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a4465ed9-c5e9-455b-82f1-1388b4f7732d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.171160 May 23 13:11:40.580420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 524/1040] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:40 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:41.597619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:41.598416 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:41.600987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=8 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:41.608205 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:41.608931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:41.610877 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:41.615708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:41.771133 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:41.779163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d93e6d27-a59c-4e06-b852-02a6a2bc8797 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.186325 May 23 13:11:41.779741 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 517/1041] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:41 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:42.796421 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:42.797015 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:42.804197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:42.804648 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:42.805422 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:42.805858 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:42.936740 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:42.944367 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fcc69aa7-a648-431e-bcfb-79ad3f7e68b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.152111 May 23 13:11:42.944923 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 525/1042] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:42 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:43.960532 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:43.961150 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:43.967915 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:43.968362 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:43.969122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:43.969799 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:44.096215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:44.103507 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-be051285-06d7-4101-8c6a-91e470cfd22a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.146831 May 23 13:11:44.104070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 518/1043] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:43 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:45.120673 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:45.121296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:45.128639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:45.131402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:45.131402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:45.131402 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:45.257846 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:45.265433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-000864cb-00b5-4223-a013-db1d7c3c4320 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.148645 May 23 13:11:45.266634 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 526/1044] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:45 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:46.286646 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:46.287193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:46.293847 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:46.294322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:46.295080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:46.295514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:46.476755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:46.484166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d9d1809e-5bcb-4f68-8bd2-2b6aa4e1470b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.203056 May 23 13:11:46.484748 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 519/1045] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:46 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:47.501281 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:47.501963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:47.504328 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:47.509191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:47.509601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:47.510420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:47.510916 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:47.513489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:47.755808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:47.763973 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-baad9f52-2d69-4ff4-a045-1f9363a926e1 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.266926 May 23 13:11:47.773845 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 527/1046] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:47 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:48.002720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-41fd7a99-4293-4fde-919d-b424770258dd req-2dbd771b-ba62-487e-bf6a-dd789658270a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:48.013080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-41fd7a99-4293-4fde-919d-b424770258dd req-2dbd771b-ba62-487e-bf6a-dd789658270a service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:48.013658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-41fd7a99-4293-4fde-919d-b424770258dd req-2dbd771b-ba62-487e-bf6a-dd789658270a service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:48.033548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-41fd7a99-4293-4fde-919d-b424770258dd req-2dbd771b-ba62-487e-bf6a-dd789658270a service nova] Creating event network-changed:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:11:48.039215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-41fd7a99-4293-4fde-919d-b424770258dd req-2dbd771b-ba62-487e-bf6a-dd789658270a service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042004 May 23 13:11:48.039939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 520/1047] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:11:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:11:48.782424 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:48.783032 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:48.790510 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:48.790955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:48.791715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:48.792146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:48.923377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:48.930813 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c3cc2c9b-cea6-468b-be34-f466de7f7cc2 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.153521 May 23 13:11:48.931368 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 528/1048] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:48 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:49.948204 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:49.948894 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:49.950813 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=52,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:49.956183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:49.956659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:49.957429 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:49.957864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:50.103921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:50.111816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-41bcd578-f9dc-48bb-8c63-a07cdd913776 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.168336 May 23 13:11:50.112383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 521/1049] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:49 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:51.128009 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:51.128624 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:51.135147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:51.135600 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:51.136364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:51.136806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:51.265798 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:51.274606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3e6484dd-4663-4531-9fe8-3a8c4bc58d79 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.150552 May 23 13:11:51.275164 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 529/1050] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:51 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:52.291890 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:52.292510 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:52.299857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:52.300252 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:52.301028 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:52.301473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:52.436743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:52.444917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7bbfd0c5-477e-4fbd-8805-bcc7641adda4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.157062 May 23 13:11:52.445481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 522/1051] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:52 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:53.461404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:53.462042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:53.468527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:53.469037 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:53.469983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:53.470416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:53.597343 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:53.604587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a896b153-dc21-4286-88fa-60ad265ec620 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.147041 May 23 13:11:53.605249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 530/1052] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:53 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:54.429967 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-cbb42c11-4d2e-4d58-9718-bd529efb13eb req-e4ef04cb-e69f-4c6c-888f-db2bff105582 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "name": "network-vif-plugged", "status": "completed", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:54.438554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-cbb42c11-4d2e-4d58-9718-bd529efb13eb req-e4ef04cb-e69f-4c6c-888f-db2bff105582 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:54.438994 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-cbb42c11-4d2e-4d58-9718-bd529efb13eb req-e4ef04cb-e69f-4c6c-888f-db2bff105582 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:54.455233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-cbb42c11-4d2e-4d58-9718-bd529efb13eb req-e4ef04cb-e69f-4c6c-888f-db2bff105582 service nova] Creating event network-vif-plugged:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:11:54.455757 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-cbb42c11-4d2e-4d58-9718-bd529efb13eb req-e4ef04cb-e69f-4c6c-888f-db2bff105582 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029438 May 23 13:11:54.456338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 523/1053] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:11:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:54.622486 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:54.623107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:54.630176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:54.630631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:54.631410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:54.631851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:54.798197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:54.806377 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dda28570-ffb1-44f0-98db-aecbd4629510 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.187758 May 23 13:11:54.807053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 531/1054] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:54 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:55.824545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:55.825260 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:55.832104 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:55.832562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:55.833320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:55.833754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:55.958995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:55.966219 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f7c1e2ce-6c53-4026-8fa1-604fb13bb9ba tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1358 microversion: 2.1 time: 0.146275 May 23 13:11:55.966768 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 524/1055] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:55 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:56.983209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:56.983817 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:56.990884 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:56.991330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:56.992082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:56.992513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.131187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:57.138658 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d9cbfca7-06f1-4575-8142-8597de2d0ee6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1548 microversion: 2.1 time: 0.159266 May 23 13:11:57.139226 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 532/1056] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:56 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1548 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:57.155317 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:57.155936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:57.163026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:57.163481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.164245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:57.164685 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.295109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:57.302674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-851a98d7-2793-428d-8da5-f179d0f71886 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1548 microversion: 2.1 time: 0.151180 May 23 13:11:57.303238 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 525/1057] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:57 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1548 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:57.318668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:57.322157 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:57.322937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Searching by: {'deleted': False, 'project_id': '1f2fed134f274a6ebe5ffe183205e269'} {{(pid=130045) get_all /opt/stack/nova/nova/compute/api.py:2962}} May 23 13:11:57.329508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:57.330549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.331534 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:57.331959 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.421103 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.multi_cell_list [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=130045) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} May 23 13:11:57.515242 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-22567a9c-9e43-4f5a-95d4-7aff72eea7ce tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.200626 May 23 13:11:57.515825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 533/1058] 10.222.0.30 () {58 vars in 1124 bytes} [Mon May 23 13:11:57 2022] GET /compute/v2.1/servers => generated 342 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:11:57.527450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:11:57.528071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:57.535466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:57.535921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.536688 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:57.537128 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:57.668926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:11:57.680600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-557fccf4-486c-4325-b66e-d51c55307681 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1548 microversion: 2.1 time: 0.153351 May 23 13:11:57.680600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 526/1059] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:11:57 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1548 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:11:59.533549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f2f1fa93-1048-41e5-ab32-46d08867a637"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:11:59.534645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:11:59.542805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:59.543254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:11:59.544027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:11:59.544460 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:00.193851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e6b42401-5036-4510-9797-738373a7528b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.664454 May 23 13:12:00.194530 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 534/1060] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:11:59 2022] POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments => generated 194 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:05.948819 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:05.949671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:05.956451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:05.956903 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:05.957669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:05.958174 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:05.985446 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0cd7a883-5df8-41e3-89d6-2f1a7f8f4d83 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.040893 May 23 13:12:05.986157 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 527/1061] 10.222.0.30 () {60 vars in 1342 bytes} [Mon May 23 13:12:05 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:06.044067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:06.044654 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:06.051149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:06.051608 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:06.052372 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:06.052812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:06.077492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:12:06.292934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b8d01c8b-dabb-4471-b3a5-85242459abab tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 204 len: 0 microversion: 2.20 time: 0.253240 May 23 13:12:06.293522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 535/1062] 10.222.0.30 () {60 vars in 1279 bytes} [Mon May 23 13:12:06 2022] DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 23 13:12:06.302178 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:06.304871 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:06.309642 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:06.310118 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:06.310892 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:06.311331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:06.438619 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:06.447305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:06.448807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:06.449993 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e62d91d6-c1b7-4d53-a6ac-a88aecec593b tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.152240 May 23 13:12:06.450547 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 528/1063] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:06 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:07.468076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:07.468771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:07.471937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:07.476885 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:07.477334 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:07.478142 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:07.478578 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:07.615896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:07.623789 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:07.625277 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:07.626462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d6819eb9-6ff4-4562-b8f9-3037b538b3a2 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.162636 May 23 13:12:07.627026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 536/1064] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:07 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:08.644486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:08.645095 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:08.652080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:08.652526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:08.653286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:08.653717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:08.781265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:08.791966 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:08.793905 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:08.795084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2029153c-38f6-4dff-9505-08f0e2cf8438 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.154471 May 23 13:12:08.795645 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 529/1065] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:08 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:09.814182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:09.814792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:09.821289 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:09.821732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:09.822524 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:09.822964 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:09.951532 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:09.959605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:09.961074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:09.962247 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-126dac9c-2ec2-47c4-bfbf-9a6fcbd96e85 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.152279 May 23 13:12:09.962799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 537/1066] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:09 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:10.981726 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:10.982305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:10.990491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:10.990949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:10.991727 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:10.992198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:10.996551 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=78 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:11.135367 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:11.149361 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:11.150961 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:11.156592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-df064e5f-6c07-415c-9bbc-8707e91a01aa tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.175034 May 23 13:12:11.156592 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 530/1067] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:10 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:12.172230 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:12.172842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:12.180082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:12.180535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:12.181305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:12.181819 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:12.357504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:12.365497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:12.367028 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:12.368227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6abe144f-02e8-4ac8-8d0a-3f2a7d49c024 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1725 microversion: 2.20 time: 0.200066 May 23 13:12:12.368779 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 538/1068] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:12 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1725 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:12.675712 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-a931eb98-9d55-4376-aa30-7cd1f08a7dd1 req-5c6d7e53-7b01-4d44-919d-6e4c8226e421 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e959d5ca-3d5d-466d-b0bb-4d50913b4a28", "name": "network-vif-deleted", "tag": "eccadb59-cfe4-41e7-ad44-d4c6052db9fa"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:12.684169 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a931eb98-9d55-4376-aa30-7cd1f08a7dd1 req-5c6d7e53-7b01-4d44-919d-6e4c8226e421 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:12.684612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a931eb98-9d55-4376-aa30-7cd1f08a7dd1 req-5c6d7e53-7b01-4d44-919d-6e4c8226e421 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:12.695858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-a931eb98-9d55-4376-aa30-7cd1f08a7dd1 req-5c6d7e53-7b01-4d44-919d-6e4c8226e421 service nova] Creating event network-vif-deleted:eccadb59-cfe4-41e7-ad44-d4c6052db9fa for instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 on hv-cinder-maste May 23 13:12:12.700816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-a931eb98-9d55-4376-aa30-7cd1f08a7dd1 req-5c6d7e53-7b01-4d44-919d-6e4c8226e421 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029423 May 23 13:12:12.701407 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 531/1069] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:13.023720 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:13.024288 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:13.030307 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:13.030755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:13.031520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:13.031955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:13.169438 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:13.178389 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-40c6b6e3-201e-4e5a-9cc4-e43e333e9759 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1594 microversion: 2.1 time: 0.158762 May 23 13:12:13.179127 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 539/1070] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:13 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1594 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:13.387689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:13.388525 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:13.396415 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:13.397101 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:13.398166 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:13.398648 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:13.464349 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:13.472669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:13.475501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:13.475501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-85016353-5733-4c11-8b63-fbc60e862f03 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1657 microversion: 2.20 time: 0.092586 May 23 13:12:13.476050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 532/1071] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:13 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1657 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:14.493672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:14.494346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:14.500843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:14.502535 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:14.503371 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:14.503881 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:14.519573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:14.581898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:14.589487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:14.591009 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:14.592141 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-32f47aca-623f-4b8d-87b3-8b34875533e9 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1657 microversion: 2.20 time: 0.102493 May 23 13:12:14.592689 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 540/1072] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:14 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1657 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:15.494360 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-937ff708-5b9b-42c5-9dff-1ae5583e3c65 req-0a6f450c-57d4-4b08-aa77-e407862abb01 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:15.500663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-937ff708-5b9b-42c5-9dff-1ae5583e3c65 req-0a6f450c-57d4-4b08-aa77-e407862abb01 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:15.501103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-937ff708-5b9b-42c5-9dff-1ae5583e3c65 req-0a6f450c-57d4-4b08-aa77-e407862abb01 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:15.513514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-937ff708-5b9b-42c5-9dff-1ae5583e3c65 req-0a6f450c-57d4-4b08-aa77-e407862abb01 service nova] Creating event network-changed:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:12:15.519575 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-937ff708-5b9b-42c5-9dff-1ae5583e3c65 req-0a6f450c-57d4-4b08-aa77-e407862abb01 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031245 May 23 13:12:15.527695 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 533/1073] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:15.610285 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:15.611067 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:15.617675 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:15.621906 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:15.621906 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:15.621906 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:15.685478 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:15.695356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:15.697951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:15.699130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b2838307-ee1b-4326-87d5-edcb745ade01 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1657 microversion: 2.20 time: 0.092802 May 23 13:12:15.700088 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 541/1074] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:15 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1657 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:16.723154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:16.723154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:16.731477 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:16.732059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:16.733051 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:16.733489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:16.771903 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:16.772605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:16.773054 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lazy-loading 'fault' on Instance uuid e959d5ca-3d5d-466d-b0bb-4d50913b4a28 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:12:16.858003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:16.868006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:16.871132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f78ab5496c1e4008b83fa3866601c06c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5b4d791d4aca45429094aa0fea349041', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:16.871132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b580f1bf-1927-4b54-a9b7-5f1c4c9b3a57 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 200 len: 1371 microversion: 2.20 time: 0.156256 May 23 13:12:16.871689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 534/1075] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:16 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 1371 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 13:12:17.889473 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:17.894404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:17.898212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:17.898659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:17.899419 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:17.899848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:17.913575 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] HTTP exception thrown: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Request method failure captured: May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 HTTP/1.1 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4WxL9Arqy9CJ88sxiUU1WzkGM6LpcxnqseuKE-TVe7S8w8RyNkHgHIjvL1lYkLK5m9PUAwdK4xH_MK7SQup-GZh75PYIGdv1gv9irohgp-YR6Taun_pGQg5-984lJupPA5HhECuSDlDKy6qX8vNz_jyEpvna6TYgladVilrQILHuFE May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Openstack-Nova-Api-Version: 2.20 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:12:17.915275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5b4d791d4aca45429094aa0fea349041"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: f78ab5496c1e4008b83fa3866601c06c May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:17.917979 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'e959d5ca-3d5d-466d-b0bb-4d50913b4a28'} May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:17.919468 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:17.921799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:17.923908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Returning 404 to user: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:17.923908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-568e480b-b0d4-40ea-bb47-8510a35aa96c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 404 len: 111 microversion: 2.20 time: 0.031081 May 23 13:12:17.923908 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 542/1076] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:17 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 13:12:17.925549 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e8b0bc97-e30d-49dc-a343-a2830d75d72c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '.version_select at 0x7f9b12fcd670>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:17.941439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1067996940"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:17.941999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:17.947963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:17.948408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:17.949165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:17.949601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:18.025335 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e8b0bc97-e30d-49dc-a343-a2830d75d72c tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1941422353" status: 204 len: 0 microversion: 2.20 time: 0.103544 May 23 13:12:18.026063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 535/1077] 10.222.0.30 () {60 vars in 1261 bytes} [Mon May 23 13:12:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1941422353 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 23 13:12:18.079959 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.network.security_group_api [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Adding security group e2399ae2-7e86-4d7e-b329-d3a4cb618298 to port 666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 May 23 13:12:18.537265 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-42604a04-581f-4b2e-8ca4-a4a3fe6b3acc tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/action" status: 202 len: 0 microversion: 2.1 time: 0.599610 May 23 13:12:18.537892 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 543/1078] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:12:17 2022] POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/action => generated 0 bytes in 601 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:12:18.546136 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:18.546767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:18.557419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:18.557909 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:18.558681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:18.559122 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:18.565051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-5d934a2c-74cd-423f-bb57-2fb3b1b910df req-5bdca3f7-3b89-4174-8a6e-a7e23fe6da56 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:18.574478 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-5d934a2c-74cd-423f-bb57-2fb3b1b910df req-5bdca3f7-3b89-4174-8a6e-a7e23fe6da56 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:18.575214 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-5d934a2c-74cd-423f-bb57-2fb3b1b910df req-5bdca3f7-3b89-4174-8a6e-a7e23fe6da56 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:18.605490 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-5d934a2c-74cd-423f-bb57-2fb3b1b910df req-5bdca3f7-3b89-4174-8a6e-a7e23fe6da56 service nova] Creating event network-changed:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:12:18.624765 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-5d934a2c-74cd-423f-bb57-2fb3b1b910df req-5bdca3f7-3b89-4174-8a6e-a7e23fe6da56 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069021 May 23 13:12:18.625764 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 544/1079] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:12:18.847324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:18.865116 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f3cbfed4-3a99-49e9-805d-12285fd2f87f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1760 microversion: 2.1 time: 0.319050 May 23 13:12:18.874933 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 536/1080] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:18 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1760 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:20.207312 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:20.207886 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:20.209787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=41,nova_api:UPDATE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:20.214646 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:20.215090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:20.215852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:20.216283 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:20.229146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] HTTP exception thrown: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Request method failure captured: May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 HTTP/1.1 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4WxL9Arqy9CJ88sxiUU1WzkGM6LpcxnqseuKE-TVe7S8w8RyNkHgHIjvL1lYkLK5m9PUAwdK4xH_MK7SQup-GZh75PYIGdv1gv9irohgp-YR6Taun_pGQg5-984lJupPA5HhECuSDlDKy6qX8vNz_jyEpvna6TYgladVilrQILHuFE May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Openstack-Nova-Api-Version: 2.20 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:12:20.230801 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5b4d791d4aca45429094aa0fea349041"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: f78ab5496c1e4008b83fa3866601c06c May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.232459 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'e959d5ca-3d5d-466d-b0bb-4d50913b4a28'} May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:20.233594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.235082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:20.236655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Returning 404 to user: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:20.236655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c560fbb0-77ed-4c47-a60a-b5e524c22731 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 404 len: 111 microversion: 2.20 time: 0.030296 May 23 13:12:20.236655 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 545/1081] 10.222.0.30 () {60 vars in 1279 bytes} [Mon May 23 13:12:20 2022] DELETE /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 13:12:20.240675 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:20.241281 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: e959d5ca-3d5d-466d-b0bb-4d50913b4a28] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:20.243690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=35,nova_api:DELETE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:20.248391 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:20.248834 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:20.249585 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:20.250098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:20.263808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] HTTP exception thrown: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Request method failure captured: May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 HTTP/1.1 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WxL9Arqy9CJ88sxiUU1WzkGM6LpcxnqseuKE-TVe7S8w8RyNkHgHIjvL1lYkLK5m9PUAwdK4xH_MK7SQup-GZh75PYIGdv1gv9irohgp-YR6Taun_pGQg5-984lJupPA5HhECuSDlDKy6qX8vNz_jyEpvna6TYgladVilrQILHuFE May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Openstack-Nova-Api-Version: 2.20 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:12:20.265424 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5b4d791d4aca45429094aa0fea349041"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: f78ab5496c1e4008b83fa3866601c06c May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.267184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'e959d5ca-3d5d-466d-b0bb-4d50913b4a28'} May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:20.268223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. May 23 13:12:20.270464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:20.271837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Returning 404 to user: Instance e959d5ca-3d5d-466d-b0bb-4d50913b4a28 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:20.271837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c203bb21-8754-4e42-a723-d6b9487327f7 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28" status: 404 len: 111 microversion: 2.20 time: 0.030136 May 23 13:12:20.271837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 537/1082] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:20 2022] GET /compute/v2.1/servers/e959d5ca-3d5d-466d-b0bb-4d50913b4a28 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 13:12:25.727080 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:25.727563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:25.734291 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:25.734741 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:25.735511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:25.735942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:25.748932 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] HTTP exception thrown: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Request method failure captured: May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 HTTP/1.1 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4WxL9Arqy9CJ88sxiUU1WzkGM6LpcxnqseuKE-TVe7S8w8RyNkHgHIjvL1lYkLK5m9PUAwdK4xH_MK7SQup-GZh75PYIGdv1gv9irohgp-YR6Taun_pGQg5-984lJupPA5HhECuSDlDKy6qX8vNz_jyEpvna6TYgladVilrQILHuFE May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Openstack-Nova-Api-Version: 2.20 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:12:25.750482 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5b4d791d4aca45429094aa0fea349041"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: f78ab5496c1e4008b83fa3866601c06c May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.752269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'd16a2136-f506-4eb4-8f03-5dadb2a6b515'} May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:25.753228 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.754808 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:25.756358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Returning 404 to user: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:25.756358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8b7245be-c4d2-4838-8b3d-f6e586506497 tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 404 len: 111 microversion: 2.20 time: 0.029642 May 23 13:12:25.756358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 546/1083] 10.222.0.30 () {60 vars in 1279 bytes} [Mon May 23 13:12:25 2022] DELETE /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 13:12:25.760132 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:25.760740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] [instance: d16a2136-f506-4eb4-8f03-5dadb2a6b515] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:25.767663 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:25.768109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:25.768864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:25.769302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:25.783438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] HTTP exception thrown: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Request method failure captured: May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 HTTP/1.1 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WxL9Arqy9CJ88sxiUU1WzkGM6LpcxnqseuKE-TVe7S8w8RyNkHgHIjvL1lYkLK5m9PUAwdK4xH_MK7SQup-GZh75PYIGdv1gv9irohgp-YR6Taun_pGQg5-984lJupPA5HhECuSDlDKy6qX8vNz_jyEpvna6TYgladVilrQILHuFE May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Openstack-Nova-Api-Version: 2.20 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:12:25.784951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_5b4d791d4aca45429094aa0fea349041"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/5b4d791d4aca45429094aa0fea349041"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 5b4d791d4aca45429094aa0fea349041 May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-329250058 May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: f78ab5496c1e4008b83fa3866601c06c May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeShelveTestJSON-329250058-project May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.786641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'd16a2136-f506-4eb4-8f03-5dadb2a6b515'} May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:25.787447 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. May 23 13:12:25.789568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:25.791100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] Returning 404 to user: Instance d16a2136-f506-4eb4-8f03-5dadb2a6b515 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:25.791100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ea4a6ef5-30a5-4708-aed1-c12ef09d9f4e tempest-AttachVolumeShelveTestJSON-329250058 tempest-AttachVolumeShelveTestJSON-329250058-project] 10.222.0.30 "GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515" status: 404 len: 111 microversion: 2.20 time: 0.030056 May 23 13:12:25.791100 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 538/1084] 10.222.0.30 () {60 vars in 1276 bytes} [Mon May 23 13:12:25 2022] GET /compute/v2.1/servers/d16a2136-f506-4eb4-8f03-5dadb2a6b515 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 13:12:32.564246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:32.564791 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:32.574457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:32.575513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:32.576621 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:32.577521 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:32.628924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:12:32.751721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-14b34c5b-b287-47c5-8ae0-54d5663919de req-a46eab10-f267-4e11-a07b-04f07797e3d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd2f1c08-5bfb-462a-9253-2c3376d9dbd2", "tag": "e5d22321-630c-4fb7-97e9-d2c4e994d264"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:32.760987 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-14b34c5b-b287-47c5-8ae0-54d5663919de req-a46eab10-f267-4e11-a07b-04f07797e3d5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:32.761438 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-14b34c5b-b287-47c5-8ae0-54d5663919de req-a46eab10-f267-4e11-a07b-04f07797e3d5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:32.773721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-14b34c5b-b287-47c5-8ae0-54d5663919de req-a46eab10-f267-4e11-a07b-04f07797e3d5 service nova] Creating event network-changed:e5d22321-630c-4fb7-97e9-d2c4e994d264 for instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 on hv-cinder-maste May 23 13:12:32.779866 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-14b34c5b-b287-47c5-8ae0-54d5663919de req-a46eab10-f267-4e11-a07b-04f07797e3d5 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.217434 May 23 13:12:32.780887 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 539/1085] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:32.982594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-935884db-dd38-4876-8fa1-8f9977556352 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "DELETE /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 204 len: 0 microversion: 2.1 time: 0.422058 May 23 13:12:32.983349 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 547/1086] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:12:32 2022] DELETE /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:12:32.994542 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:32.996230 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:33.004030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:33.004728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:33.005844 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:33.007330 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:33.157538 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:33.167110 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-244c6d15-50e9-410c-a9c3-dff6e035b142 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1578 microversion: 2.1 time: 0.178753 May 23 13:12:33.177721 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 540/1087] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:32 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1578 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:34.186235 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:34.186955 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:34.193610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:34.194717 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:34.195685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:34.196119 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:34.448155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:34.455844 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-64724256-8a44-4a29-be01-e6775a413018 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1578 microversion: 2.1 time: 0.273939 May 23 13:12:34.456411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 548/1088] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:34 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1578 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:35.473931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:35.474504 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:35.482662 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:35.483432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:35.484439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:35.485044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:35.561144 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:35.571876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a6c868a4-36fe-4005-8a24-abb29dfcd83d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1393 microversion: 2.1 time: 0.101850 May 23 13:12:35.597297 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 541/1089] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:35 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1393 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:35.665363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-8f08bd43-7a98-49da-91e7-6fed524ddcc3 req-dee6a56d-5214-4e93-957d-15c562c588a7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd2f1c08-5bfb-462a-9253-2c3376d9dbd2", "name": "network-vif-deleted", "tag": "e5d22321-630c-4fb7-97e9-d2c4e994d264"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:35.673974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-8f08bd43-7a98-49da-91e7-6fed524ddcc3 req-dee6a56d-5214-4e93-957d-15c562c588a7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:35.674481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-8f08bd43-7a98-49da-91e7-6fed524ddcc3 req-dee6a56d-5214-4e93-957d-15c562c588a7 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:35.687182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-8f08bd43-7a98-49da-91e7-6fed524ddcc3 req-dee6a56d-5214-4e93-957d-15c562c588a7 service nova] Creating event network-vif-deleted:e5d22321-630c-4fb7-97e9-d2c4e994d264 for instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 on hv-cinder-maste May 23 13:12:35.697182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-8f08bd43-7a98-49da-91e7-6fed524ddcc3 req-dee6a56d-5214-4e93-957d-15c562c588a7 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036024 May 23 13:12:35.706117 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 549/1090] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:12:36.597552 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:36.598195 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:36.608991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:36.608991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:36.608991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:36.608991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:36.683985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:36.701377 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-75d4e65e-7e36-4fc1-9e62-07e25a434a27 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1226 microversion: 2.1 time: 0.106207 May 23 13:12:36.703186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 542/1091] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:36 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1226 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:37.227422 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d629b1ab-3ad7-4d8b-87a5-f0511230941d tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: .version_select at 0x7f9b1bf22f70>, body: {"keypair": {"name": "tempest-keypair-85072002", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE2pVo72lGKOk8Z9XzToIz8SmKhoj4FIf9u/uX1uOTHAIm7a84rwZyj3j8x0vQXEGVgqo+MFCGzcpzPNCbUN9LTl6NFTA5FfNlrWvYvk9LnFrZYZ9ZiJCD6tCnvBhRQuNw=="}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:37.241902 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d629b1ab-3ad7-4d8b-87a5-f0511230941d tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:12:37.245705 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d629b1ab-3ad7-4d8b-87a5-f0511230941d tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for user fe5a3e6e6a0b4509b92cdcd54da0f4fe and project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:12:37.356835 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d629b1ab-3ad7-4d8b-87a5-f0511230941d tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.188688 May 23 13:12:37.357429 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 550/1092] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:12:37 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:37.718568 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:37.719185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:37.725804 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:37.726286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:37.727072 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:37.727595 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:37.789460 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:37.797453 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0bbc3430-2193-4730-a87b-a611227e5f80 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1226 microversion: 2.1 time: 0.082703 May 23 13:12:37.799331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 543/1093] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:37 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1226 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:38.816489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:38.817271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:38.825669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:38.826341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:38.830205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:38.830205 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:38.891860 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:38.901254 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cca9e7ac-1b7a-41a0-9d96-27e63a3a3779 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1226 microversion: 2.1 time: 0.088882 May 23 13:12:38.902104 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 551/1094] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:38 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:39.558486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-173948782", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4643b097-3168-4b82-93f6-b8d1f98fc5a8"}], "security_groups": [{"name": "tempest-securitygroup--425208750"}], "key_name": "tempest-keypair-85072002", "user_data": "}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:39.672451 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] validate_networks() for [('4643b097-3168-4b82-93f6-b8d1f98fc5a8', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:12:39.924154 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:39.924843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:39.932346 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:39.932873 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:39.934066 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:39.934241 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:40.280828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:40.289079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ff52135c-9e68-4cb8-86bb-349302a452a2 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1226 microversion: 2.1 time: 0.374602 May 23 13:12:40.291090 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 552/1095] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:39 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1226 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:40.407738 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:12:40.408161 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:12:40.408637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:12:40.409378 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:12:40.409794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:12:40.420690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:40.421139 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:40.421965 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:40.422400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:40.441437 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:12:40.449203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for user fe5a3e6e6a0b4509b92cdcd54da0f4fe and project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:12:40.459979 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:12:40.462404 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:12:40.627876 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5740ffaa-8e07-44bc-aaf9-ec92e3c175bd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.074399 May 23 13:12:40.628672 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 544/1096] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:12:39 2022] POST /compute/v2.1/servers => generated 401 bytes in 1076 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:12:40.639565 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-59d43765-cbb6-494e-aa01-0a50446004c0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:40.640909 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-59d43765-cbb6-494e-aa01-0a50446004c0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:40.715439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-59d43765-cbb6-494e-aa01-0a50446004c0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:40.735838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-59d43765-cbb6-494e-aa01-0a50446004c0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1235 microversion: 2.1 time: 0.100178 May 23 13:12:40.749442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 553/1097] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:40 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1235 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:41.308479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:41.309368 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:41.320117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:41.321288 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:41.322341 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:41.323220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:41.328629 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=62 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:41.358751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:41.359206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:41.359659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lazy-loading 'fault' on Instance uuid dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:12:41.417575 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:41.429288 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-24b9c12b-2f8d-4cc5-a433-1eda9e3048f9 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 200 len: 1231 microversion: 2.1 time: 0.125534 May 23 13:12:41.430790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 545/1098] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:41 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 1231 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:41.759225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1267277b-3959-4aa1-8eb5-868316b1eafd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:41.759898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1267277b-3959-4aa1-8eb5-868316b1eafd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:41.814146 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1267277b-3959-4aa1-8eb5-868316b1eafd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:41.823027 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1267277b-3959-4aa1-8eb5-868316b1eafd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1235 microversion: 2.1 time: 0.067741 May 23 13:12:41.824267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 554/1099] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:41 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1235 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:42.448678 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:42.449279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: dd2f1c08-5bfb-462a-9253-2c3376d9dbd2] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:42.457080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:42.457550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:42.458340 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:42.458790 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:42.473038 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] HTTP exception thrown: Instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 could not be found. May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Request method failure captured: May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 HTTP/1.1 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WcprMIovHsqBKyUVjs61zB0JHcTlcKlthe0PJsLBmFXx2-NmHfOhkAWRQ2FhhVNCOeow5WG3z8p74dYLMbclpdujgDEiBMpyPbs8Mnd1bWc18nvhNt6DgJNwzZ1GN_L5Jbuv8SD2jrQMOf6_1IzrjEk9G8QFIu7_8buaPgmv53Bs0 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:12:42.474761 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: c6a6096760734ac49ce2f26c6065b701 May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 could not be found. May 23 13:12:42.476530 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'dd2f1c08-5bfb-462a-9253-2c3376d9dbd2'} May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 could not be found. May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:42.478383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 could not be found. May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 could not be found. May 23 13:12:42.479940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:42.481264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Returning 404 to user: Instance dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:42.481264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-84ff6395-5310-4800-800a-5c697fc55279 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2" status: 404 len: 111 microversion: 2.1 time: 0.032068 May 23 13:12:42.481264 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 546/1100] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:42 2022] GET /compute/v2.1/servers/dd2f1c08-5bfb-462a-9253-2c3376d9dbd2 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:12:42.839978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:42.840722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:42.847109 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:42.851207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:42.851207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:42.851207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:42.913489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:42.921769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dafc44df-4a58-4b8a-bae8-23df48dc0d3f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1287 microversion: 2.1 time: 0.086186 May 23 13:12:42.922360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 555/1101] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:42 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1287 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:43.939031 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:43.939861 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:43.947319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:43.947694 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:43.948408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:43.948852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:44.160276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:44.169301 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-029e1186-5915-451f-aebf-226c4a9823ad tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.234081 May 23 13:12:44.170222 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 547/1102] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:43 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:44.847987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-2edf1034-1e12-4bc8-8613-e7e2ab068352 req-8e11142f-2bdb-4b58-994b-e23012186c6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a123fe36-b018-410e-a6a1-13c25f9713c1", "tag": "dd19a70c-27a3-47de-a03d-931ffbf395d7"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:44.857995 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-2edf1034-1e12-4bc8-8613-e7e2ab068352 req-8e11142f-2bdb-4b58-994b-e23012186c6a service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:44.858321 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-2edf1034-1e12-4bc8-8613-e7e2ab068352 req-8e11142f-2bdb-4b58-994b-e23012186c6a service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:44.861203 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:44.875408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-2edf1034-1e12-4bc8-8613-e7e2ab068352 req-8e11142f-2bdb-4b58-994b-e23012186c6a service nova] Creating event network-changed:dd19a70c-27a3-47de-a03d-931ffbf395d7 for instance a123fe36-b018-410e-a6a1-13c25f9713c1 on hv-cinder-maste May 23 13:12:44.877014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-2edf1034-1e12-4bc8-8613-e7e2ab068352 req-8e11142f-2bdb-4b58-994b-e23012186c6a service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033682 May 23 13:12:44.877622 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 556/1103] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:45.196131 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:45.196736 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:45.204417 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:45.204955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:45.205805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:45.206337 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:45.461256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:45.475501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6ca8b7f1-808a-4486-8326-57cf77a87d64 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.282940 May 23 13:12:45.476988 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 548/1104] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:45 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:46.124259 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-5fd608cf-549a-4eac-80b8-ce33526342b7 req-9f6f8e88-b136-47a9-b930-d513d6ab5ffe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1ded1afb-ad2c-4b62-a675-0b685e5fa30a", "tag": "bfe7ce80-1e0d-4d50-b178-8416be1a709c"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:46.132604 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5fd608cf-549a-4eac-80b8-ce33526342b7 req-9f6f8e88-b136-47a9-b930-d513d6ab5ffe service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.133053 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5fd608cf-549a-4eac-80b8-ce33526342b7 req-9f6f8e88-b136-47a9-b930-d513d6ab5ffe service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.134602 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:46.135891 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:46.146416 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-5fd608cf-549a-4eac-80b8-ce33526342b7 req-9f6f8e88-b136-47a9-b930-d513d6ab5ffe service nova] Creating event network-changed:bfe7ce80-1e0d-4d50-b178-8416be1a709c for instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a on hv-cinder-maste May 23 13:12:46.152564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-5fd608cf-549a-4eac-80b8-ce33526342b7 req-9f6f8e88-b136-47a9-b930-d513d6ab5ffe service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031836 May 23 13:12:46.153157 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 549/1105] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:12:46.155566 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.156061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.156954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.157628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.195568 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:12:46.498310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:46.499082 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:46.506714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.507531 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.508311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.508752 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.583310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ad088534-735a-4473-8ca8-b6cda605061c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "DELETE /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 204 len: 0 microversion: 2.1 time: 0.466172 May 23 13:12:46.584165 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 557/1106] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:12:46 2022] DELETE /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 0 bytes in 469 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:12:46.593713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:46.594579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:46.601273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.602761 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.603036 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:46.603583 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:46.688732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:46.700924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-4ab398d2-4f35-47d2-a9ae-ccc2ca8dc5bb tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.209411 May 23 13:12:46.701514 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 550/1107] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:46 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:46.776428 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:46.784828 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1ed045ab-907b-4353-9782-5cf1522c6ac4 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1578 microversion: 2.1 time: 0.195848 May 23 13:12:46.785418 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 558/1108] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:46 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1578 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:47.717895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:47.718759 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:47.725198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:47.725660 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:47.726485 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:47.726935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:47.803043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:47.803792 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:47.810465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:47.811112 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:47.812077 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:47.812978 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:47.896356 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:47.905497 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-272899be-d6ba-43a1-b981-4d8ab0ac8c5e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.191543 May 23 13:12:47.906363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 551/1109] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:47 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:47.961450 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:47.969864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f5d89d61-7343-4859-87b3-41fcafe83c14 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1578 microversion: 2.1 time: 0.170590 May 23 13:12:47.970451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 559/1110] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:47 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1578 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:48.922198 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:48.922812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:48.930751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:48.930751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:48.930751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:48.930751 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:48.987523 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:48.988340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:48.995355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:48.999061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:48.999061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:48.999061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:49.235181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:49.241164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b3663b69-01ed-4d76-95ed-fdd642ee344b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.322794 May 23 13:12:49.241996 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 552/1111] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:48 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:49.298278 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:49.306768 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4b87245c-108c-4cd4-bcca-02e5c72b9243 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1578 microversion: 2.1 time: 0.322444 May 23 13:12:49.307543 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 560/1112] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:48 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1578 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:49.740137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-5f3a753c-d9bb-4c81-914e-5fc0cd59d701 req-6d6e57e8-7cdb-43f1-b472-6bcf6ffb6767 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1ded1afb-ad2c-4b62-a675-0b685e5fa30a", "name": "network-vif-deleted", "tag": "bfe7ce80-1e0d-4d50-b178-8416be1a709c"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:49.753566 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5f3a753c-d9bb-4c81-914e-5fc0cd59d701 req-6d6e57e8-7cdb-43f1-b472-6bcf6ffb6767 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:49.754491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5f3a753c-d9bb-4c81-914e-5fc0cd59d701 req-6d6e57e8-7cdb-43f1-b472-6bcf6ffb6767 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:49.767909 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-5f3a753c-d9bb-4c81-914e-5fc0cd59d701 req-6d6e57e8-7cdb-43f1-b472-6bcf6ffb6767 service nova] Creating event network-vif-deleted:bfe7ce80-1e0d-4d50-b178-8416be1a709c for instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a on hv-cinder-maste May 23 13:12:49.773719 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-5f3a753c-d9bb-4c81-914e-5fc0cd59d701 req-6d6e57e8-7cdb-43f1-b472-6bcf6ffb6767 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037154 May 23 13:12:49.774740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 553/1113] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:50.258628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:50.259361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:50.266292 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:50.266915 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:50.267706 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:50.268144 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:50.325019 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:50.325634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:50.332846 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:50.333305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:50.334126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:50.334572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:50.424955 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6a6096760734ac49ce2f26c6065b701', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c7ef8e9bc68d4e60b8993eac9f160a65', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:50.428463 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:50.433410 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d4dd337c-3379-439e-b0ae-0ae2fbc1132b tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 200 len: 1226 microversion: 2.1 time: 0.112923 May 23 13:12:50.442748 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:50.443768 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 554/1114] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:50 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 1226 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:50.444703 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7ed9af5a-4014-4bc4-a3af-230afdce2b95 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.189973 May 23 13:12:50.445260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 561/1115] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:50 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:51.450982 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:51.451527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 1ded1afb-ad2c-4b62-a675-0b685e5fa30a] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:51.453244 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:51.458789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:51.459236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:51.459994 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:51.460427 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:51.460562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:51.461130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:51.468374 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:51.468867 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:51.469691 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:51.470208 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:51.476183 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] HTTP exception thrown: Instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a could not be found. May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Request method failure captured: May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a HTTP/1.1 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WcprMIovHsqBKyUVjs61zB0JHcTlcKlthe0PJsLBmFXx2-NmHfOhkAWRQ2FhhVNCOeow5WG3z8p74dYLMbclpdujgDEiBMpyPbs8Mnd1bWc18nvhNt6DgJNwzZ1GN_L5Jbuv8SD2jrQMOf6_1IzrjEk9G8QFIu7_8buaPgmv53Bs0 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:12:51.478652 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: c6a6096760734ac49ce2f26c6065b701 May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a could not be found. May 23 13:12:51.480610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '1ded1afb-ad2c-4b62-a675-0b685e5fa30a'} May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a could not be found. May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:51.481490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a could not be found. May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a could not be found. May 23 13:12:51.482958 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:51.484170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Returning 404 to user: Instance 1ded1afb-ad2c-4b62-a675-0b685e5fa30a could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:51.484170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ef242160-ca75-4dd4-b784-9d834c30b96c tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a" status: 404 len: 111 microversion: 2.1 time: 0.033554 May 23 13:12:51.484170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 555/1116] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:51 2022] GET /compute/v2.1/servers/1ded1afb-ad2c-4b62-a675-0b685e5fa30a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:12:51.625163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:51.634755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-066e894e-ea3e-4fdd-b3f3-291d7613a1aa tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.177200 May 23 13:12:51.635528 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 562/1117] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:51 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:12:51.916618 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-a48cc6b2-c885-4e10-8600-7299ce78dc3f req-2d0eac6e-a0f9-4e8a-99a7-bc91bdd91ae5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a123fe36-b018-410e-a6a1-13c25f9713c1", "name": "network-vif-plugged", "status": "completed", "tag": "dd19a70c-27a3-47de-a03d-931ffbf395d7"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:51.925545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a48cc6b2-c885-4e10-8600-7299ce78dc3f req-2d0eac6e-a0f9-4e8a-99a7-bc91bdd91ae5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:51.926019 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-a48cc6b2-c885-4e10-8600-7299ce78dc3f req-2d0eac6e-a0f9-4e8a-99a7-bc91bdd91ae5 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:51.938354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-a48cc6b2-c885-4e10-8600-7299ce78dc3f req-2d0eac6e-a0f9-4e8a-99a7-bc91bdd91ae5 service nova] Creating event network-vif-plugged:dd19a70c-27a3-47de-a03d-931ffbf395d7 for instance a123fe36-b018-410e-a6a1-13c25f9713c1 on hv-cinder-maste May 23 13:12:51.943299 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-a48cc6b2-c885-4e10-8600-7299ce78dc3f req-2d0eac6e-a0f9-4e8a-99a7-bc91bdd91ae5 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030441 May 23 13:12:51.943889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 556/1118] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:52.037787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:52.038389 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:52.045021 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=8 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:52.048420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:52.053153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:52.054048 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:52.054567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:52.070645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] HTTP exception thrown: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Request method failure captured: May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 HTTP/1.1 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4WcprMIovHsqBKyUVjs61zB0JHcTlcKlthe0PJsLBmFXx2-NmHfOhkAWRQ2FhhVNCOeow5WG3z8p74dYLMbclpdujgDEiBMpyPbs8Mnd1bWc18nvhNt6DgJNwzZ1GN_L5Jbuv8SD2jrQMOf6_1IzrjEk9G8QFIu7_8buaPgmv53Bs0 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:12:52.072337 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: c6a6096760734ac49ce2f26c6065b701 May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.074620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '7064ed49-237d-441d-a087-70b195245624'} May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:52.075410 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.076851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:12:52.078350 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Returning 404 to user: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:52.078350 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b1984b66-7638-4b67-ac76-2155ebb9d377 tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "DELETE /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 404 len: 111 microversion: 2.1 time: 0.039780 May 23 13:12:52.078350 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 563/1119] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:12:52 2022] DELETE /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 13:12:52.083148 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:52.083924 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] [instance: 7064ed49-237d-441d-a087-70b195245624] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:52.090827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:52.091281 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:52.096033 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:52.096033 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:52.106362 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] HTTP exception thrown: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Request method failure captured: May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 HTTP/1.1 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4WcprMIovHsqBKyUVjs61zB0JHcTlcKlthe0PJsLBmFXx2-NmHfOhkAWRQ2FhhVNCOeow5WG3z8p74dYLMbclpdujgDEiBMpyPbs8Mnd1bWc18nvhNt6DgJNwzZ1GN_L5Jbuv8SD2jrQMOf6_1IzrjEk9G8QFIu7_8buaPgmv53Bs0 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:12:52.107973 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c7ef8e9bc68d4e60b8993eac9f160a65"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: c7ef8e9bc68d4e60b8993eac9f160a65 May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-TestVolumeBootPattern-2089392200 May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: c6a6096760734ac49ce2f26c6065b701 May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-TestVolumeBootPattern-2089392200-project May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.109626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '7064ed49-237d-441d-a087-70b195245624'} May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:12:52.110483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. May 23 13:12:52.111749 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:12:52.113010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Returning 404 to user: Instance 7064ed49-237d-441d-a087-70b195245624 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:12:52.113010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e5cb5d44-cbb6-47f2-9a65-f7448d8c4d9d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624" status: 404 len: 111 microversion: 2.1 time: 0.030488 May 23 13:12:52.113010 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 557/1120] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:52 2022] GET /compute/v2.1/servers/7064ed49-237d-441d-a087-70b195245624 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:12:52.654176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:52.654890 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:52.661324 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:52.661803 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:52.668335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:52.668335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:52.793406 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:52.801310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-13f6a097-613b-40bb-bf21-9694b1e70864 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.151310 May 23 13:12:52.801884 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 564/1121] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:52 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:53.652381 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-46032a21-5baa-4074-8829-1aaad30abb5d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] Calling method '.version_select at 0x7f9b1806aca0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:53.745184 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-46032a21-5baa-4074-8829-1aaad30abb5d tempest-TestVolumeBootPattern-2089392200 tempest-TestVolumeBootPattern-2089392200-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1848200419" status: 202 len: 0 microversion: 2.1 time: 0.096319 May 23 13:12:53.746295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 558/1122] 10.222.0.30 () {58 vars in 1262 bytes} [Mon May 23 13:12:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1848200419 => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:12:53.821380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:53.822268 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:53.829130 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:53.829587 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:53.830389 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:53.831012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:53.971370 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:53.979331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-80929398-0d40-47dd-945d-9a3fbe73c9ba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1360 microversion: 2.1 time: 0.161810 May 23 13:12:53.979901 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 565/1123] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:53 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1360 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:54.998455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:55.000351 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:55.009130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:55.010612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:55.011587 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:55.012185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:55.222922 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:12:55.231435 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9eb63070-adac-493d-94bf-99106cb96f8b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1547 microversion: 2.1 time: 0.238704 May 23 13:12:55.232567 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 559/1124] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:12:54 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1547 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:12:55.252568 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:12:55.253770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:12:55.262121 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:55.262869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:55.263903 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:55.264549 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:55.390245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-798983b2-3766-4368-b6e7-dc188f759d48 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.141639 May 23 13:12:55.390834 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 566/1125] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:12:55 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-interface => generated 285 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:12:56.592215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-5b98692c-30bd-4cd3-8b47-daa2f97a098e req-b12ee0ff-8706-4bdd-b9d9-a9a55e75fead service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a123fe36-b018-410e-a6a1-13c25f9713c1", "tag": "dd19a70c-27a3-47de-a03d-931ffbf395d7"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:12:56.601603 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5b98692c-30bd-4cd3-8b47-daa2f97a098e req-b12ee0ff-8706-4bdd-b9d9-a9a55e75fead service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:12:56.602080 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-5b98692c-30bd-4cd3-8b47-daa2f97a098e req-b12ee0ff-8706-4bdd-b9d9-a9a55e75fead service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:12:56.618518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-5b98692c-30bd-4cd3-8b47-daa2f97a098e req-b12ee0ff-8706-4bdd-b9d9-a9a55e75fead service nova] Creating event network-changed:dd19a70c-27a3-47de-a03d-931ffbf395d7 for instance a123fe36-b018-410e-a6a1-13c25f9713c1 on hv-cinder-maste May 23 13:12:56.624354 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-5b98692c-30bd-4cd3-8b47-daa2f97a098e req-b12ee0ff-8706-4bdd-b9d9-a9a55e75fead service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036343 May 23 13:12:56.628217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 560/1126] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:12:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:13:19.697479 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:13:19.698326 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:13:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:13:33.246303 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:33.247569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:33.251507 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:13:33.252436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:33.258586 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:33.264096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:33.264096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:33.264096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:33.466605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b719f928-15e1-4bf3-b490-412961c98f4c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/action" status: 202 len: 0 microversion: 2.1 time: 0.221014 May 23 13:13:33.468967 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 567/1127] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:13:33 2022] POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/action => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:13:33.475866 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:33.476580 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:33.479565 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:33.480215 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=63 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:33.485901 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:33.488910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:33.488910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:33.488910 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:33.678001 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:33.687639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b0cc30ef-6fe1-4cce-b680-1c75e9b50d30 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1752 microversion: 2.1 time: 0.215941 May 23 13:13:33.688234 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 561/1128] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:33 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:34.706580 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:34.707278 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:34.713562 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:34.714069 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:34.714825 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:34.715286 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:34.861749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:34.869650 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-81a5e5ee-db59-4682-98de-4f58000ed429 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.168528 May 23 13:13:34.870240 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 568/1129] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:34 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:35.888242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:35.888833 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:35.895340 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:35.895812 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:35.896561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:35.897256 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:36.037400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:36.045474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-50e284df-ab07-4a62-be68-da2017c7d98a tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.161997 May 23 13:13:36.046374 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 562/1130] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:35 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:37.062957 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:37.063513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:37.069596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:37.070084 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:37.070839 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:37.071301 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:37.211225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:37.219778 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-86a11010-3bbc-41d3-8b3d-4596bddaea60 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.161087 May 23 13:13:37.220340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 569/1131] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:37 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:38.237061 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:38.237664 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:38.244042 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:38.244486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:38.245235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:38.245666 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:38.392905 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:38.401387 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f515b49a-d636-4eaf-a655-528390cd0d7e tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.168140 May 23 13:13:38.401983 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 563/1132] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:38 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:39.420202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:39.420927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:39.427522 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:39.427966 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:39.428716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:39.429143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:39.570579 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:39.578816 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4df6f7c9-816b-40cc-b923-14f68f4dc03b tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.162533 May 23 13:13:39.579373 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 570/1133] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:39 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:40.596846 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:40.597396 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:40.603985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:40.604432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:40.605196 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:40.605630 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:40.749490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:40.757829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-219d41b3-52e0-46c5-a56c-a840f0724ff9 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.164873 May 23 13:13:40.758450 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 564/1134] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:40 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:41.774849 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:41.775456 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:41.782699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:41.783140 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:41.783893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:41.784325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:41.926963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:41.935985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-0154b58c-6c3c-4de4-8b41-228c7108bac7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.164238 May 23 13:13:41.936781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 571/1135] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:41 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:42.954996 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:42.955640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:42.962314 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:42.962764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:42.963537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:42.963974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:43.118456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:43.127397 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-641eea68-4ac6-4212-b458-51d5e09ea57c tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.177483 May 23 13:13:43.128091 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 565/1136] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:42 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:44.145442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:44.146081 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:44.153082 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:44.154099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:44.155050 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:44.155489 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:44.296073 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:44.304028 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f4bc3b71-a95d-4dac-a61d-dfa17fdcb548 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1757 microversion: 2.1 time: 0.162448 May 23 13:13:44.306152 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 572/1137] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:44 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:13:45.320637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:13:45.321322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:13:45.328151 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:45.328650 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:45.329416 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:13:45.329851 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:13:45.471739 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:13:45.479639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a33c589a-e8fa-450f-b0d5-6b9bfc50e1af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1760 microversion: 2.1 time: 0.163033 May 23 13:13:45.480202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 566/1138] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:13:45 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1760 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:12.419504 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:12.420083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:12.422051 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=12 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:12.422725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:12.427513 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:12.427954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:12.428733 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:12.429161 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:12.452045 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3cb628d2-30c8-408e-bbec-2bca5e9206b0 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/ips" status: 200 len: 145 microversion: 2.1 time: 0.036866 May 23 13:14:12.452637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 573/1139] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:14:12 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/ips => generated 145 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:14:14.855498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4f58d4d0-ce17-442c-b415-9f0579b2368d"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:14.856607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:14.859378 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=12 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:14.860027 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=36 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:14.864691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:14.865167 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:14.865974 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:14.866508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:15.647449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2f673765-08f9-4450-811f-19ce011f8d85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.795974 May 23 13:14:15.648044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 567/1140] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:14:14 2022] POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-volume_attachments => generated 194 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:14:24.187958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b85eb8a5-878c-4a14-b274-0b3c173b3175 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:24.195361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b85eb8a5-878c-4a14-b274-0b3c173b3175 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:24.195820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b85eb8a5-878c-4a14-b274-0b3c173b3175 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:24.255011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b85eb8a5-878c-4a14-b274-0b3c173b3175 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Going to try to stop instance {{(pid=130045) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 23 13:14:24.474820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b85eb8a5-878c-4a14-b274-0b3c173b3175 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action" status: 202 len: 0 microversion: 2.1 time: 0.290838 May 23 13:14:24.475597 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 574/1141] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:14:24 2022] POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action => generated 0 bytes in 292 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:14:24.484286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:24.484990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:24.491712 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:24.492286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:24.492931 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:24.493400 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:24.627860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:24.635750 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9c7968c9-d7e9-4ac2-93c2-61fa39e1c109 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.155864 May 23 13:14:24.636308 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 568/1142] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:24 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:25.654701 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:25.655972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:25.662152 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:25.662594 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:25.663340 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:25.663769 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:25.791977 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:25.800158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dad93952-e0f5-4a87-a1ae-a090c1196ad5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.150168 May 23 13:14:25.800745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 575/1143] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:25 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:26.816935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:26.817539 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:26.824191 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:26.824636 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:26.825676 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:26.825813 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:26.961040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:26.970318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0a5d3118-480e-4fa5-97c9-9884347ffe77 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.157091 May 23 13:14:26.971242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 569/1144] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:26 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:27.988186 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:27.988776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:27.995295 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:27.995749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:27.996508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:27.996947 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:28.122137 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:28.130401 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c8993af0-7be0-42af-b737-9768287024df tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.146022 May 23 13:14:28.130966 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 576/1145] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:27 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:29.148102 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:29.148911 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:29.155767 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:29.156227 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:29.156995 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:29.157435 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:29.282302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:29.290107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d8721017-2860-4865-8d8f-5676316c2790 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.146222 May 23 13:14:29.290667 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 570/1146] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:29 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:30.306560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:30.307195 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:30.314071 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:30.314516 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:30.315267 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:30.315698 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:30.444007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:30.452665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-67d8cc72-f1cc-4f65-9648-bea536e3c792 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.149978 May 23 13:14:30.453327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 577/1147] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:30 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:31.469840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:31.470486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:31.476991 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:31.477480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:31.478278 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:31.478718 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:31.605481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:31.613711 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-492bd21a-c64c-42a1-9c9e-fb20a7dd2283 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.147776 May 23 13:14:31.614293 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 571/1148] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:31 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:32.632080 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:32.632686 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:32.639127 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:32.640026 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:32.640809 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:32.641246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:32.778860 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:32.786700 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-29fd899b-c342-4a1f-a4ad-cd4af046fe59 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.158434 May 23 13:14:32.787253 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 578/1149] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:32 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:33.804464 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:33.805078 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:33.811753 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:33.812201 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:33.812960 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:33.813398 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:33.940163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:33.948041 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-86918990-aca8-439c-bddb-eeede95d7396 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.147531 May 23 13:14:33.948550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 572/1150] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:33 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:34.965814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:34.966420 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:34.973294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:34.973732 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:34.974511 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:34.974938 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:35.105591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:35.114381 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1c2ecfb4-80f8-4655-8a67-315be43dc86f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1720 microversion: 2.1 time: 0.152508 May 23 13:14:35.114942 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 579/1151] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:34 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1720 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:36.131550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:36.132156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:36.138622 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:36.139071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:36.140460 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:36.140989 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:36.267423 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:36.275522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-e3d1a9c2-e9b5-4b59-89df-23d9bf682ff8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1697 microversion: 2.1 time: 0.147809 May 23 13:14:36.276083 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 573/1152] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:36 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1697 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:36.291659 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6acea27e-8018-4933-b3e3-81485aacdebe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:36.298674 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6acea27e-8018-4933-b3e3-81485aacdebe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:36.299116 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6acea27e-8018-4933-b3e3-81485aacdebe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:36.321002 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6acea27e-8018-4933-b3e3-81485aacdebe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Going to try to start instance {{(pid=130045) start /opt/stack/nova/nova/compute/api.py:2730}} May 23 13:14:36.480349 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6acea27e-8018-4933-b3e3-81485aacdebe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action" status: 202 len: 0 microversion: 2.1 time: 0.192217 May 23 13:14:36.480881 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 580/1153] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:14:36 2022] POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:14:36.490282 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:36.490888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:36.496824 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:36.497363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:36.498152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:36.498591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:36.660603 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:36.669683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8888a423-bb78-4adc-b582-c15baa15cb7f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1706 microversion: 2.1 time: 0.183260 May 23 13:14:36.670287 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 574/1154] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:36 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1706 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:37.689457 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:37.690187 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:37.696606 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:37.697056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:37.697818 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:37.698308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:37.825272 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:37.835215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-15c002d6-c593-4056-af61-5f2e04cd0e3a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1706 microversion: 2.1 time: 0.151110 May 23 13:14:37.835787 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 581/1155] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:37 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1706 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:38.852865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:38.853481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:38.860250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:38.860700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:38.861462 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:38.861921 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:38.994336 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:39.003097 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c903c32f-e7f8-4f1a-a409-4a096b8aef98 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1710 microversion: 2.1 time: 0.154097 May 23 13:14:39.003658 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 575/1156] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:38 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1710 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:43.642559 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1067996940"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:43.643093 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:43.649715 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:43.650180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:43.650934 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:43.651363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:43.775341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.network.security_group_api [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Removing security group e2399ae2-7e86-4d7e-b329-d3a4cb618298 from port 666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 May 23 13:14:44.194397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c0141b6d-1753-43b9-8584-a3569f4044a5 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/action" status: 202 len: 0 microversion: 2.1 time: 0.556522 May 23 13:14:44.198212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 582/1157] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:14:43 2022] POST /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/action => generated 0 bytes in 558 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 13:14:44.207768 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-25917f0f-57ab-40db-b4c1-d6cbcc8b1155 req-4c846510-530d-4931-aa49-31205d817ed8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:44.239323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-25917f0f-57ab-40db-b4c1-d6cbcc8b1155 req-4c846510-530d-4931-aa49-31205d817ed8 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:44.239323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-25917f0f-57ab-40db-b4c1-d6cbcc8b1155 req-4c846510-530d-4931-aa49-31205d817ed8 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:44.252888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-25917f0f-57ab-40db-b4c1-d6cbcc8b1155 req-4c846510-530d-4931-aa49-31205d817ed8 service nova] Creating event network-changed:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:14:44.273476 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-25917f0f-57ab-40db-b4c1-d6cbcc8b1155 req-4c846510-530d-4931-aa49-31205d817ed8 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070253 May 23 13:14:44.274434 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 576/1158] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:14:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:14:45.477006 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-42454ed7-a6f8-4446-9eba-cf77aad20df3 req-9246fea5-3fa5-4aeb-9086-e5613e992af3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:45.481085 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=21 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:45.484877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:45.488245 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:45.491542 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:45.492271 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:45.493000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-42454ed7-a6f8-4446-9eba-cf77aad20df3 req-9246fea5-3fa5-4aeb-9086-e5613e992af3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:45.493141 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:45.493459 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-42454ed7-a6f8-4446-9eba-cf77aad20df3 req-9246fea5-3fa5-4aeb-9086-e5613e992af3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:45.493602 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:45.498094 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:45.506917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-42454ed7-a6f8-4446-9eba-cf77aad20df3 req-9246fea5-3fa5-4aeb-9086-e5613e992af3 service nova] Creating event network-changed:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:14:45.512548 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-42454ed7-a6f8-4446-9eba-cf77aad20df3 req-9246fea5-3fa5-4aeb-9086-e5613e992af3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039420 May 23 13:14:45.513498 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 577/1159] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:14:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:14:45.595649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lazy-loading 'services' on Instance uuid b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:14:45.779664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1458edbc-1097-4d48-bccd-f040d39e05f0 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments/f2f1fa93-1048-41e5-ab32-46d08867a637" status: 202 len: 0 microversion: 2.1 time: 0.311132 May 23 13:14:45.784351 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 583/1160] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:14:45 2022] DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments/f2f1fa93-1048-41e5-ab32-46d08867a637 => generated 0 bytes in 314 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:14:45.791321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:45.792782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:45.801728 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:45.802248 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:45.803059 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:45.803508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:45.839303 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8b8f9d50-3350-4446-8bc3-b2f062002d44 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053306 May 23 13:14:45.843468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 578/1161] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:14:45 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:14:46.853981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:46.855115 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:46.861269 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:46.865627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:46.865627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:46.865627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:46.894628 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3818d6cb-56f9-415c-83f8-1195ae2636af tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046485 May 23 13:14:46.895305 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 584/1162] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:14:46 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:14:47.906206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:47.907202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:47.914115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:47.914562 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:47.915318 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:47.915753 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:47.943273 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b4886e86-6e95-434f-8366-5bd5b748966d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.040994 May 23 13:14:47.943818 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 579/1163] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:14:47 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:14:47.954417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:47.954896 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:47.961158 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:47.961599 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:47.962380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:47.962809 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:48.036569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] HTTP exception thrown: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 is not attached to volume f2f1fa93-1048-41e5-ab32-46d08867a637 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Request method failure captured: May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments/f2f1fa93-1048-41e5-ab32-46d08867a637 HTTP/1.1 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4fu5bWqvTF2uipdlVfXwMLnmNUjA7WECfO0PhPNGUjNStlpftqguJBPLZ8EvN7_FWeURBE332-qhZK8wou_1ylJk8NIukXO5agk2nyskmUkhwFl877d5mDKC0F3s7y6x6zU0iLJy98SCJkuBq-0e8r_BjX2U3saDg-MHjogzfHsWwE May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 1f2fed134f274a6ebe5ffe183205e269 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-TestMinimumBasicScenario-744924327 May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:14:48.039018 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-TestMinimumBasicScenario-744924327 May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 1f2fed134f274a6ebe5ffe183205e269 May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-TestMinimumBasicScenario-744924327 May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-TestMinimumBasicScenario-744924327-project May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 20d9973f32f847a6a54a60405beabdbf May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-TestMinimumBasicScenario-744924327-project May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:14:48.040763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 is not attached to volume f2f1fa93-1048-41e5-ab32-46d08867a637 May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'server_id': 'b6ee588e-bc42-4a89-b3c1-4cfe56da2e39', 'id': 'f2f1fa93-1048-41e5-ab32-46d08867a637'} May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id f2f1fa93-1048-41e5-ab32-46d08867a637. May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:14:48.041716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 is not attached to volume f2f1fa93-1048-41e5-ab32-46d08867a637 May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 is not attached to volume f2f1fa93-1048-41e5-ab32-46d08867a637 May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:14:48.043143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Returning 404 to user: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 is not attached to volume f2f1fa93-1048-41e5-ab32-46d08867a637 {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:14:48.044185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-dea0c0d7-a531-467d-bc9a-c62fa5c0f6f6 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments/f2f1fa93-1048-41e5-ab32-46d08867a637" status: 404 len: 154 microversion: 2.1 time: 0.090784 May 23 13:14:48.044185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 585/1164] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:14:47 2022] DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39/os-volume_attachments/f2f1fa93-1048-41e5-ab32-46d08867a637 => generated 154 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:14:49.306794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:49.307278 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:49.314209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:49.314651 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:49.315403 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:49.315832 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:49.339900 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:14:49.631291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b2d66d94-ba70-49bc-9363-7c82c52f9de7 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 204 len: 0 microversion: 2.1 time: 0.327966 May 23 13:14:49.634041 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 580/1165] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:14:49 2022] DELETE /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:14:49.640382 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:49.640989 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:49.648657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:49.649096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:49.649847 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:49.650313 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:49.783435 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:49.790862 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5f844610-5c2b-4ea7-862b-6beebcae555f tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1554 microversion: 2.1 time: 0.154485 May 23 13:14:49.791417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 586/1166] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:49 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1554 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:50.808653 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:50.809442 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:50.815816 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:50.816265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:50.817021 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:50.817460 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:50.950780 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:50.958202 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-11b2facd-a5aa-4bc7-a955-6c0815dfcd6d tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1554 microversion: 2.1 time: 0.153735 May 23 13:14:50.958764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 581/1167] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:50 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1554 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:51.975822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:51.976465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:51.982954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:51.983397 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:51.984155 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:51.984592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:52.012394 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:52.206869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:52.214259 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-92f97e42-ab46-4431-a32d-2b8921705c75 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1554 microversion: 2.1 time: 0.242292 May 23 13:14:52.214831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 587/1168] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:51 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1554 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:52.516827 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-0a07f022-6ccb-4781-add6-33bc0552fcd4 req-edfba19e-bb9e-4771-989b-39080a19fa43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6ee588e-bc42-4a89-b3c1-4cfe56da2e39", "name": "network-vif-deleted", "tag": "666df93a-3cd5-4d9a-b131-dd41c5c3e3c5"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:14:52.525589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-0a07f022-6ccb-4781-add6-33bc0552fcd4 req-edfba19e-bb9e-4771-989b-39080a19fa43 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:52.528037 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-0a07f022-6ccb-4781-add6-33bc0552fcd4 req-edfba19e-bb9e-4771-989b-39080a19fa43 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:52.537747 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-0a07f022-6ccb-4781-add6-33bc0552fcd4 req-edfba19e-bb9e-4771-989b-39080a19fa43 service nova] Creating event network-vif-deleted:666df93a-3cd5-4d9a-b131-dd41c5c3e3c5 for instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 on hv-cinder-maste May 23 13:14:52.542702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-0a07f022-6ccb-4781-add6-33bc0552fcd4 req-edfba19e-bb9e-4771-989b-39080a19fa43 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029598 May 23 13:14:52.545325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 582/1169] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:14:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:14:53.231000 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:53.231589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:53.237973 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:53.238692 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:53.239168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:53.239596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:53.269361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:53.269802 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:53.270275 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lazy-loading 'fault' on Instance uuid b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:14:53.310376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20d9973f32f847a6a54a60405beabdbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f2fed134f274a6ebe5ffe183205e269', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:14:53.318408 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-691d5ce1-b24a-4048-a3f9-60220c4cccde tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 200 len: 1348 microversion: 2.1 time: 0.091250 May 23 13:14:53.322355 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 588/1170] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:53 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 1348 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:14:54.337047 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:54.337434 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [instance: b6ee588e-bc42-4a89-b3c1-4cfe56da2e39] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:14:54.343845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:54.345064 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:54.345838 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:14:54.346302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:14:54.359394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] HTTP exception thrown: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 could not be found. May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Request method failure captured: May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 HTTP/1.1 May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4fu5bWqvTF2uipdlVfXwMLnmNUjA7WECfO0PhPNGUjNStlpftqguJBPLZ8EvN7_FWeURBE332-qhZK8wou_1ylJk8NIukXO5agk2nyskmUkhwFl877d5mDKC0F3s7y6x6zU0iLJy98SCJkuBq-0e8r_BjX2U3saDg-MHjogzfHsWwE May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 1f2fed134f274a6ebe5ffe183205e269 May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-TestMinimumBasicScenario-744924327 May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:14:54.361155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/1f2fed134f274a6ebe5ffe183205e269"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-TestMinimumBasicScenario-744924327 May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 1f2fed134f274a6ebe5ffe183205e269 May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-TestMinimumBasicScenario-744924327 May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-TestMinimumBasicScenario-744924327-project May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 20d9973f32f847a6a54a60405beabdbf May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-TestMinimumBasicScenario-744924327-project May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 could not be found. May 23 13:14:54.362858 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'b6ee588e-bc42-4a89-b3c1-4cfe56da2e39'} May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 could not be found. May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:14:54.363805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 could not be found. May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 could not be found. May 23 13:14:54.365098 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:14:54.372210 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Returning 404 to user: Instance b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:14:54.372210 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8fae2708-bb18-4d70-8ed2-c815453c45b4 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39" status: 404 len: 111 microversion: 2.1 time: 0.030320 May 23 13:14:54.372210 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 583/1171] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:14:54 2022] GET /compute/v2.1/servers/b6ee588e-bc42-4a89-b3c1-4cfe56da2e39 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:14:54.372501 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f467a197-dd94-4139-8afb-f584dc1b5b28 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] Calling method '.version_select at 0x7f9b1bf9b280>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:14:54.374527 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR oslo.messaging._drivers.impl_rabbit [None req-f467a197-dd94-4139-8afb-f584dc1b5b28 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [91422227-95e9-4650-ad38-b478e8ff007f] AMQP server on 10.222.0.30:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 13:14:55.384040 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [None req-f467a197-dd94-4139-8afb-f584dc1b5b28 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] [91422227-95e9-4650-ad38-b478e8ff007f] Reconnected to AMQP server on 10.222.0.30:5672 via [amqp] client with port 45716. May 23 13:14:55.390079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=35 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:55.494866 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f467a197-dd94-4139-8afb-f584dc1b5b28 tempest-TestMinimumBasicScenario-744924327 tempest-TestMinimumBasicScenario-744924327-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1658194789" status: 202 len: 0 microversion: 2.1 time: 1.127256 May 23 13:14:55.497936 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 589/1172] 10.222.0.30 () {58 vars in 1271 bytes} [Mon May 23 13:14:54 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1658194789 => generated 0 bytes in 1129 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:15:10.786508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:15:10.786945 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:15:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:15:34.279365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:34.280093 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:34.282142 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:15:34.282848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=37,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:15:34.288266 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:34.290914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:34.290914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:34.290914 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:34.593723 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lazy-loading 'services' on Instance uuid a123fe36-b018-410e-a6a1-13c25f9713c1 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:15:34.803641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-71f51bb5-1501-4f1f-ba9b-cbe3be5abf02 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-volume_attachments/4f58d4d0-ce17-442c-b415-9f0579b2368d" status: 202 len: 0 microversion: 2.1 time: 0.524879 May 23 13:15:34.803641 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 584/1173] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:15:34 2022] DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-volume_attachments/4f58d4d0-ce17-442c-b415-9f0579b2368d => generated 0 bytes in 526 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:15:35.912877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=10 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:15:35.913699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=2,nova_api:DELETE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:15:35.917710 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4b1152de-1619-4a85-9c89-c84134c7b7f6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:15:35.924605 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b1152de-1619-4a85-9c89-c84134c7b7f6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:35.932218 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4b1152de-1619-4a85-9c89-c84134c7b7f6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:35.947541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4b1152de-1619-4a85-9c89-c84134c7b7f6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Going to try to stop instance {{(pid=130045) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 23 13:15:36.143567 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4b1152de-1619-4a85-9c89-c84134c7b7f6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action" status: 202 len: 0 microversion: 2.1 time: 0.231467 May 23 13:15:36.144553 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 590/1174] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:15:35 2022] POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action => generated 0 bytes in 234 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:15:36.153586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:36.154320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:36.161250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:36.161700 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:36.162493 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:36.162923 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:36.297017 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:36.304745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-013f654f-37c5-4cee-b4f3-c68ef7034f66 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.155386 May 23 13:15:36.305634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 585/1175] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:36 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:37.329831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:37.330542 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:37.337390 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:37.337831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:37.338609 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:37.339042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:37.565699 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:37.573812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-652e0a13-3227-4c88-abf2-8666f2848689 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.249006 May 23 13:15:37.574404 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 591/1176] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:37 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:38.591254 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:38.591842 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:38.598323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:38.598775 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:38.600032 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:38.600468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:38.732653 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:38.740035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2c18b619-03ad-489e-bb18-b1ca39a97655 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.152645 May 23 13:15:38.741265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 586/1177] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:38 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:39.758059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:39.758664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:39.765142 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:39.765584 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:39.766363 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:39.766793 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:39.895739 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:39.903708 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d87a9e48-0ecf-49f6-a1e2-875bf4c02c44 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.149711 May 23 13:15:39.904259 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 592/1178] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:39 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:40.921378 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:40.922012 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:40.928528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:40.928972 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:40.929727 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:40.930180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:41.054807 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:41.062193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-f3e38ee4-e2e5-4c3b-bb70-a42244c7d281 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.144701 May 23 13:15:41.062784 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 587/1179] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:40 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:42.079821 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:42.080439 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:42.086991 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:42.087441 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:42.088206 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:42.088645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:42.222707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:42.230079 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bc22cd9b-5a23-43b0-9487-7aba40d5797b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.154100 May 23 13:15:42.230635 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 593/1180] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:42 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:43.247734 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:43.248345 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:43.254732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:43.255179 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:43.255938 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:43.256913 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:43.387743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:43.395235 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3397d479-f442-439c-9cac-8661ba461f9a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.151369 May 23 13:15:43.396052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 588/1181] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:43 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:44.413610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:44.414247 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:44.420646 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:44.421092 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:44.421850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:44.422310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:44.554851 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:44.559812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a09bb3af-e60b-4095-b3e2-9a2852231c42 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.150639 May 23 13:15:44.560369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 594/1182] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:44 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:45.577352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:45.577978 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:45.584449 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:45.584895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:45.586333 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:45.586760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:45.714193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:45.721537 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-74278200-d44a-4e52-9e72-550c20f3f3c1 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1674 microversion: 2.1 time: 0.147988 May 23 13:15:45.722063 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 589/1183] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:45 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1674 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:46.739738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:46.740735 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:46.747760 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:46.748215 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:46.748982 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:46.749426 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:46.889038 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:46.897572 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6d2ad8db-c9a6-4672-9f8b-d0b5aeaf22c6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1651 microversion: 2.1 time: 0.161689 May 23 13:15:46.898236 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 595/1184] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:46 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1651 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:46.914668 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6ac73f12-6c1a-4aae-a2cd-640f08a72553 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:15:46.921840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ac73f12-6c1a-4aae-a2cd-640f08a72553 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:46.922327 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6ac73f12-6c1a-4aae-a2cd-640f08a72553 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:46.984861 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6ac73f12-6c1a-4aae-a2cd-640f08a72553 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Going to try to start instance {{(pid=130046) start /opt/stack/nova/nova/compute/api.py:2730}} May 23 13:15:47.152745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6ac73f12-6c1a-4aae-a2cd-640f08a72553 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action" status: 202 len: 0 microversion: 2.1 time: 0.242342 May 23 13:15:47.153352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 590/1185] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:15:46 2022] POST /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/action => generated 0 bytes in 244 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:15:47.164291 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:47.164911 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:47.170965 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:47.171418 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:47.172189 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:47.172627 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:47.332352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:47.342790 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ca8255de-fcb0-4910-b332-42eeb5dd2724 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1660 microversion: 2.1 time: 0.184180 May 23 13:15:47.343837 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 596/1186] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:47 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1660 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:48.407470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:48.407470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:48.407470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:48.407470 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:48.408413 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:48.408413 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:48.501158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:48.509070 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-037bd06f-a149-49f5-b40a-af4ed0226f6f tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1660 microversion: 2.1 time: 0.150062 May 23 13:15:48.509626 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 591/1187] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:48 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1660 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:15:49.527341 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:15:49.527954 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:15:49.534664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:49.535107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:49.535856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:15:49.536286 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:15:49.670840 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:15:49.678327 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b76f77bc-f3e9-47bd-957d-7189ebe201d7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1664 microversion: 2.1 time: 0.155535 May 23 13:15:49.679488 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 597/1188] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:15:49 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1664 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:16:39.706499 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:16:39.707358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:16:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:16:52.863605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=16 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:16:52.864452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=55,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:16:52.868357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:52.869352 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:52.875247 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:52.875687 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:52.876436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:52.876870 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:52.906024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0ee54c5d-2457-4cb9-be01-580e4408a1cd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044190 May 23 13:16:52.906570 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 592/1189] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:16:52 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:16:52.961973 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=16 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:16:52.962716 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:16:52.967065 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:52.967552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:52.973207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:52.973941 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:52.974495 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:52.974920 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:53.000056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Going to try to terminate instance {{(pid=130045) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:16:53.318406 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-893b34e4-cf11-434f-baaf-bcbaf64fd611 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 204 len: 0 microversion: 2.1 time: 0.357592 May 23 13:16:53.318907 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 598/1190] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:16:52 2022] DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:16:53.328711 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:53.332634 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:53.340056 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:53.340850 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:53.341639 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:53.342117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:53.477550 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:16:53.485714 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-736f95fb-ce2e-4f7b-928e-10b507dc603e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1670 microversion: 2.1 time: 0.162639 May 23 13:16:53.486301 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 593/1191] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:16:53 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1670 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:16:54.503506 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:54.504209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:54.510755 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:54.511207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:54.512481 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:54.512919 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:54.641099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:16:54.648805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6b54763f-94a9-451d-946e-3ce85b206b1e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1670 microversion: 2.1 time: 0.149605 May 23 13:16:54.649361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 599/1192] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:16:54 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1670 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:16:55.666703 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:55.667393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:55.674502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:55.674950 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:55.675720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:55.676155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:55.888331 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:16:55.896014 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-bd0216b7-98e2-4fda-a79e-496acf891f52 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1670 microversion: 2.1 time: 0.233123 May 23 13:16:55.896584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 594/1193] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:16:55 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1670 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:16:56.522308 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-8c5dbc20-9fc8-4de2-9550-211f80c98815 req-8d8edd28-62c6-4399-88fb-f4ecd586317b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a123fe36-b018-410e-a6a1-13c25f9713c1", "name": "network-vif-deleted", "tag": "dd19a70c-27a3-47de-a03d-931ffbf395d7"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:16:56.531452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-8c5dbc20-9fc8-4de2-9550-211f80c98815 req-8d8edd28-62c6-4399-88fb-f4ecd586317b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:56.531889 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-8c5dbc20-9fc8-4de2-9550-211f80c98815 req-8d8edd28-62c6-4399-88fb-f4ecd586317b service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:56.547757 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-8c5dbc20-9fc8-4de2-9550-211f80c98815 req-8d8edd28-62c6-4399-88fb-f4ecd586317b service nova] Creating event network-vif-deleted:dd19a70c-27a3-47de-a03d-931ffbf395d7 for instance a123fe36-b018-410e-a6a1-13c25f9713c1 on hv-cinder-maste May 23 13:16:56.553472 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-8c5dbc20-9fc8-4de2-9550-211f80c98815 req-8d8edd28-62c6-4399-88fb-f4ecd586317b service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095517 May 23 13:16:56.554159 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 600/1194] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:16:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:16:56.912997 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:56.913621 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:56.920571 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:56.921040 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:56.921801 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:56.922272 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:56.955426 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:56.955874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:56.956320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.objects.instance [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lazy-loading 'fault' on Instance uuid a123fe36-b018-410e-a6a1-13c25f9713c1 {{(pid=130046) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:16:56.999888 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:16:57.007253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dfc9529c-111c-4c12-a84a-a2ad026750cf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 200 len: 1325 microversion: 2.1 time: 0.098143 May 23 13:16:57.007798 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 595/1195] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:16:56 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 1325 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:16:58.025369 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:58.026023 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:16:58.032963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:58.033415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:58.034207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:16:58.034664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:16:58.048094 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] HTTP exception thrown: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Request method failure captured: May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 HTTP/1.1 May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4hF4ZyDPnlDN9ASmMY792zN0OYK6Y1drU2nFqk0NVzaWI5z2I9j7lkgoigSVKq9VosdFCoar02lG9b3AEhGjB9W1bCcI_m3JMjSjqsUpbglkpd2vBa-iV0oVGhFl4I_pTRq_hcw0iv0Er_uhrZVRzFvswsLirgykyQWw81iHiO7-vc May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:16:58.049684 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeTestJSON-2097949922 May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: fe5a3e6e6a0b4509b92cdcd54da0f4fe May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:16:58.051452 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'a123fe36-b018-410e-a6a1-13c25f9713c1'} May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:16:58.052552 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:16:58.054227 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:16:58.055687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Returning 404 to user: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:16:58.055687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7d7350db-2729-40fa-8686-d8eacd25b857 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 404 len: 111 microversion: 2.1 time: 0.030467 May 23 13:16:58.055687 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 601/1196] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:16:58 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:16:58.060050 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6d1b75ce-c8ea-4d0f-87e2-a3f457abec72 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '.version_select at 0x7f9b12fc5dc0>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:16:58.129518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6d1b75ce-c8ea-4d0f-87e2-a3f457abec72 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-85072002" status: 202 len: 0 microversion: 2.1 time: 0.073386 May 23 13:16:58.130103 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 596/1197] 10.222.0.30 () {58 vars in 1214 bytes} [Mon May 23 13:16:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-85072002 => generated 0 bytes in 75 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:16:58.868077 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6632df03-6de5-4025-8fa4-249f09a98fbe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: .version_select at 0x7f9b12a623a0>, body: {"keypair": {"name": "tempest-keypair-259479803", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLVgrm68snk22YciWF1n1ewQ4eH/QWl3I0aO6kPipaqtP3CQR3wFftNvbarcQQKooN6MK+ON6AcHOCT0UIA6yd1OJ1s1hMFaWgSX28cqUInKFHwheScQsDCw6h1sWFVTcg=="}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:16:58.882178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-6632df03-6de5-4025-8fa4-249f09a98fbe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:16:58.885990 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-6632df03-6de5-4025-8fa4-249f09a98fbe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for user fe5a3e6e6a0b4509b92cdcd54da0f4fe and project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'key_pairs'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:16:58.976390 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6632df03-6de5-4025-8fa4-249f09a98fbe tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.112078 May 23 13:16:58.976940 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 602/1198] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:16:58 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:17:01.158814 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-689254966", "imageRef": "b7729a52-809a-4821-8cf7-c74021950998", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4643b097-3168-4b82-93f6-b8d1f98fc5a8"}], "security_groups": [{"name": "tempest-securitygroup--1673261102"}], "key_name": "tempest-keypair-259479803", "user_data": "}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:17:01.259155 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.network.neutron [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] validate_networks() for [('4643b097-3168-4b82-93f6-b8d1f98fc5a8', None, None, None, None, None)] {{(pid=130046) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:17:01.618469 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Flavor limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:17:01.618878 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Image limits 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:17:01.619353 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Flavor pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:17:01.619763 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Image pref 0:0:0 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:17:01.620170 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.virt.hardware [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130046) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:17:01.629358 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:01.629809 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:01.630637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:01.631073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:01.647446 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:17:01.651325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Getting quotas for user fe5a3e6e6a0b4509b92cdcd54da0f4fe and project 11bdfcfe4dbb42d3bb07585d85bff52a. Resources: {'instances', 'ram', 'cores'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:17:01.662432 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Going to run 1 instances... {{(pid=130046) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:17:01.664685 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b7729a52-809a-4821-8cf7-c74021950998',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130046) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:17:01.783495 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-42242ea7-bc5c-4812-a719-c44f1957992c tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.629608 May 23 13:17:01.784139 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 597/1199] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:17:01 2022] POST /compute/v2.1/servers => generated 402 bytes in 631 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:17:01.795143 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-8518fa62-0d13-4d3c-9f76-d45a26b03904 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:01.795734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-8518fa62-0d13-4d3c-9f76-d45a26b03904 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:01.855958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-8518fa62-0d13-4d3c-9f76-d45a26b03904 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:01.863217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8518fa62-0d13-4d3c-9f76-d45a26b03904 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1236 microversion: 2.1 time: 0.073024 May 23 13:17:01.863831 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 603/1200] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:01 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1236 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:02.882490 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:02.883108 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:02.889970 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:02.890421 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:02.891189 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:02.891628 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:02.955465 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:02.963309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7f7a83c1-1d39-41c1-8eba-3304511f9de7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1232 microversion: 2.1 time: 0.085148 May 23 13:17:02.963884 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 598/1201] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:02 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1232 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:03.985251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:03.985774 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:03.992299 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:03.992747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:03.993507 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:03.993966 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:04.063952 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:04.071306 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ca20ac25-a261-402d-83b2-9ad486eefedf tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1294 microversion: 2.1 time: 0.090105 May 23 13:17:04.071864 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 604/1202] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:03 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1294 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:05.088927 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:05.089640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:05.096088 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:05.096542 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:05.097305 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:05.097759 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:05.225265 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-bf3cc46c-5bd8-4f07-b0bb-8da0646a976a req-bf37d3ff-2974-4023-8fac-80ff4cfdaecf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c120d794-5327-4846-9553-2b508248915e", "tag": "67b2df5a-1169-4fd1-bce0-0716babb1144"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:17:05.237193 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-bf3cc46c-5bd8-4f07-b0bb-8da0646a976a req-bf37d3ff-2974-4023-8fac-80ff4cfdaecf service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:05.237492 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-bf3cc46c-5bd8-4f07-b0bb-8da0646a976a req-bf37d3ff-2974-4023-8fac-80ff4cfdaecf service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:05.256734 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-bf3cc46c-5bd8-4f07-b0bb-8da0646a976a req-bf37d3ff-2974-4023-8fac-80ff4cfdaecf service nova] Creating event network-changed:67b2df5a-1169-4fd1-bce0-0716babb1144 for instance c120d794-5327-4846-9553-2b508248915e on hv-cinder-maste May 23 13:17:05.262125 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-bf3cc46c-5bd8-4f07-b0bb-8da0646a976a req-bf37d3ff-2974-4023-8fac-80ff4cfdaecf service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040387 May 23 13:17:05.263330 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 605/1203] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:17:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:17:05.290466 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:05.301516 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2ce4ca73-8e61-465d-b27f-8aebe1cf9a03 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.216439 May 23 13:17:05.302247 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 599/1204] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:05 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:06.320323 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:06.321231 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:06.327554 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:06.328005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:06.328848 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:06.329288 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:06.462167 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:06.469842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7fe871ad-dc57-4de5-bfbb-08e2a0f35564 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.154222 May 23 13:17:06.470433 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 606/1205] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:06 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:07.486773 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:07.487320 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:07.493702 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:07.494174 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:07.494929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:07.495359 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:07.622223 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:07.629394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-9a944165-c9e4-4546-9d76-6057397682a5 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.146625 May 23 13:17:07.629977 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 600/1206] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:07 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:17:08.647153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:08.647771 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:08.654191 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:08.654646 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:08.655412 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:08.655853 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:08.788149 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:08.796107 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-30cecb9f-154f-4bbf-88ff-c52247ab4f85 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.153279 May 23 13:17:08.796665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 607/1207] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:08 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:09.813489 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:09.814137 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:09.820574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:09.821024 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:09.821783 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:09.822245 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:09.948560 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:09.956610 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6778b561-acac-4fb6-9d98-6db13adda820 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.148018 May 23 13:17:09.957164 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 601/1208] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:09 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:10.973494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:10.974123 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:10.981148 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:10.981597 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:10.982383 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:10.982981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:11.111493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:11.118894 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3081e617-be54-490f-aa3b-06d93a81dc6a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.149221 May 23 13:17:11.119458 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 608/1209] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:10 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:11.240480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-4090ec2d-008d-4640-8c06-3fedc951adc2 req-e8073d3c-c5b1-470b-91ea-67b5680df6f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c120d794-5327-4846-9553-2b508248915e", "name": "network-vif-plugged", "status": "completed", "tag": "67b2df5a-1169-4fd1-bce0-0716babb1144"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:17:11.251039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4090ec2d-008d-4640-8c06-3fedc951adc2 req-e8073d3c-c5b1-470b-91ea-67b5680df6f8 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:11.251480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4090ec2d-008d-4640-8c06-3fedc951adc2 req-e8073d3c-c5b1-470b-91ea-67b5680df6f8 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:11.262646 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-4090ec2d-008d-4640-8c06-3fedc951adc2 req-e8073d3c-c5b1-470b-91ea-67b5680df6f8 service nova] Creating event network-vif-plugged:67b2df5a-1169-4fd1-bce0-0716babb1144 for instance c120d794-5327-4846-9553-2b508248915e on hv-cinder-maste May 23 13:17:11.268047 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-4090ec2d-008d-4640-8c06-3fedc951adc2 req-e8073d3c-c5b1-470b-91ea-67b5680df6f8 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090389 May 23 13:17:11.268868 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 602/1210] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:17:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:17:12.135766 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:12.136375 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:12.138416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:17:12.143709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:12.144417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:12.145182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:12.145615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:12.276243 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:12.284114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-650d6e47-ee22-4515-8970-4fc15f29111b tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.152204 May 23 13:17:12.284668 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 609/1211] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:12 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:13.301981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:13.302670 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:13.304633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:17:13.305332 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=6 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:17:13.310808 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:13.311279 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:13.312038 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:13.312468 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:13.444863 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:13.452683 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-52884e85-ad3f-4492-839a-eb0d93250c30 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1362 microversion: 2.1 time: 0.154994 May 23 13:17:13.453242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 603/1212] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:13 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:14.468974 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:14.469577 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:14.476071 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:14.476519 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:14.477273 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:14.477709 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:14.610729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:17:14.618114 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ff654625-18d9-46a9-8c21-7e86e03db592 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1548 microversion: 2.1 time: 0.153032 May 23 13:17:14.618669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 610/1213] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:17:14 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1548 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:17:14.634677 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:17:14.635168 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:17:14.641895 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:14.642342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:14.643109 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:14.643544 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:14.722661 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-656ac841-a6f0-4b04-b56a-f51e4ec1747a tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091746 May 23 13:17:14.723265 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 604/1214] 10.222.0.30 () {58 vars in 1274 bytes} [Mon May 23 13:17:14 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:17:15.707342 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-5f656691-ccb9-432f-9f3b-39b7d8d52b6e req-cc276d57-899a-4154-a364-11075ae9132f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c120d794-5327-4846-9553-2b508248915e", "tag": "67b2df5a-1169-4fd1-bce0-0716babb1144"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:17:15.716494 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-5f656691-ccb9-432f-9f3b-39b7d8d52b6e req-cc276d57-899a-4154-a364-11075ae9132f service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:17:15.716940 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-5f656691-ccb9-432f-9f3b-39b7d8d52b6e req-cc276d57-899a-4154-a364-11075ae9132f service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:17:15.727789 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-5f656691-ccb9-432f-9f3b-39b7d8d52b6e req-cc276d57-899a-4154-a364-11075ae9132f service nova] Creating event network-changed:67b2df5a-1169-4fd1-bce0-0716babb1144 for instance c120d794-5327-4846-9553-2b508248915e on hv-cinder-maste May 23 13:17:15.733147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-5f656691-ccb9-432f-9f3b-39b7d8d52b6e req-cc276d57-899a-4154-a364-11075ae9132f service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029461 May 23 13:17:15.733984 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 611/1215] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:17:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:18:20.379943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:20.380709 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=64 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:20.434864 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:20.435357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:20.442355 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:20.449445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:20.449445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:20.449445 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:20.466139 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b9ae9718-9582-4fcf-bd4a-fa9d9eb9b256 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/ips" status: 200 len: 146 microversion: 2.1 time: 0.088957 May 23 13:18:20.466692 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 605/1216] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:18:20 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/ips => generated 146 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:18:23.051315 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:23.052928 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:23.057243 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "703b0d07-aa8c-42fd-9387-be39254f552e"}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:18:23.058411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:23.064361 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:23.067508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:23.067508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:23.067508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:23.810011 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-54d007ba-f341-4903-b266-aaa9b15f9cab tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.760462 May 23 13:18:23.810616 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 612/1217] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:18:23 2022] POST /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments => generated 194 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:31.295689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:31.296669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:31.298912 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=4 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:31.304154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:31.304601 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:31.305357 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:31.305795 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:31.335934 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fa6b041e-42fc-4692-8389-c60efecec854 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044290 May 23 13:18:31.336481 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 606/1218] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:18:31 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:31.347436 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:31.348056 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:31.354449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:31.354893 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:31.355645 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:31.356075 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:31.384829 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-3b1c574b-9495-4296-931d-46419e90f7b8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/703b0d07-aa8c-42fd-9387-be39254f552e" status: 200 len: 194 microversion: 2.1 time: 0.042102 May 23 13:18:31.385389 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 613/1219] 10.222.0.30 () {58 vars in 1412 bytes} [Mon May 23 13:18:31 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/703b0d07-aa8c-42fd-9387-be39254f552e => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:32.947073 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6abb073f-cd0e-4621-b40b-c3bf7042536e"}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:18:32.948057 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:32.954356 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:32.954805 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:32.955557 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:32.955988 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:33.776003 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ad8ce04f-1bf3-4d70-b817-09b829606557 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "POST /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.834350 May 23 13:18:33.776640 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 607/1220] 10.222.0.30 () {60 vars in 1322 bytes} [Mon May 23 13:18:32 2022] POST /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments => generated 194 bytes in 836 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:42.309747 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:42.310731 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:42.313169 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=4 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:42.313811 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:18:42.319497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:42.324639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:42.324639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:42.324639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:42.350849 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1e73c1c7-d7c9-4f3e-a62b-940a66a68450 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.044965 May 23 13:18:42.351399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 614/1221] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:18:42 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments => generated 371 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:42.492547 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:42.493061 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:42.499268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:42.499715 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:42.500471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:42.501047 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:42.529784 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-70ffd5f1-fc76-43e4-aedf-dad7c1cef9c4 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/703b0d07-aa8c-42fd-9387-be39254f552e" status: 200 len: 194 microversion: 2.1 time: 0.042327 May 23 13:18:42.530383 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 608/1222] 10.222.0.30 () {58 vars in 1412 bytes} [Mon May 23 13:18:42 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/703b0d07-aa8c-42fd-9387-be39254f552e => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:42.540417 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:42.540904 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:42.547204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:42.553296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:42.553296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:42.553296 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:42.628352 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lazy-loading 'services' on Instance uuid c120d794-5327-4846-9553-2b508248915e {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:18:42.822287 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-50ccc44b-60b6-4720-9ab1-e05f0628709e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/703b0d07-aa8c-42fd-9387-be39254f552e" status: 202 len: 0 microversion: 2.1 time: 0.285954 May 23 13:18:42.822877 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 615/1223] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:18:42 2022] DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/703b0d07-aa8c-42fd-9387-be39254f552e => generated 0 bytes in 287 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:18:44.986526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:44.987035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:44.994087 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:44.994543 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:44.995342 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:44.995781 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:45.024465 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8f7dc6b4-91eb-468a-9191-3c848b030fca tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/6abb073f-cd0e-4621-b40b-c3bf7042536e" status: 200 len: 194 microversion: 2.1 time: 0.041982 May 23 13:18:45.025041 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 609/1224] 10.222.0.30 () {58 vars in 1412 bytes} [Mon May 23 13:18:44 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/6abb073f-cd0e-4621-b40b-c3bf7042536e => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:45.035601 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:45.036182 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:45.042539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:45.042987 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:45.043745 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:45.044180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:45.125660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lazy-loading 'services' on Instance uuid c120d794-5327-4846-9553-2b508248915e {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:18:45.336497 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-afe626c2-44ca-44d7-8b48-15ccd13ffbbd tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/6abb073f-cd0e-4621-b40b-c3bf7042536e" status: 202 len: 0 microversion: 2.1 time: 0.305398 May 23 13:18:45.337076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 616/1225] 10.222.0.30 () {58 vars in 1415 bytes} [Mon May 23 13:18:45 2022] DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments/6abb073f-cd0e-4621-b40b-c3bf7042536e => generated 0 bytes in 307 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:18:46.499691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:46.500679 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:46.507021 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:46.511473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:46.511473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:46.511473 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:46.538917 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-b1ae86bf-6895-482a-9558-143d6cda4503 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042977 May 23 13:18:46.539981 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 610/1226] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:18:46 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:18:46.648411 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:46.649469 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:46.655595 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:46.656046 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:46.656814 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:46.657252 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:46.685589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-934e49c7-55e6-41c3-ac8f-cececc791dc7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041982 May 23 13:18:46.686185 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 617/1227] 10.222.0.30 () {58 vars in 1301 bytes} [Mon May 23 13:18:46 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 13:18:46.743760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:46.744185 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:46.757290 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:46.758625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:46.759529 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:46.759963 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:46.784589 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:18:46.993690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-51987416-6c66-42dd-ab59-628fe2e356f3 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 204 len: 0 microversion: 2.1 time: 0.254303 May 23 13:18:46.994364 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 611/1228] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:18:46 2022] DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 13:18:47.006820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:47.008054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:47.014013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:47.014485 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:47.015212 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:47.015647 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:47.144764 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:18:47.153012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1723d734-3cba-4493-85f1-11d2f18e0fba tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1673 microversion: 2.1 time: 0.150179 May 23 13:18:47.153573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 618/1229] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:47 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1673 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:18:48.171316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:48.172036 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:48.178393 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:48.178835 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:48.179778 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:48.180076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:48.311276 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:18:48.318760 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c6d26270-d9f7-42d0-9b10-f2bc22e08bb8 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1673 microversion: 2.1 time: 0.151647 May 23 13:18:48.319321 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 612/1230] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:48 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1673 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:18:49.337518 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:49.338175 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:49.344640 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:49.345025 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:49.345775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:49.346229 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:49.560753 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:18:49.568100 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cc0ddf65-65bf-4893-afce-ba27377f5ef7 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1673 microversion: 2.1 time: 0.235099 May 23 13:18:49.568666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 619/1231] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:49 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1673 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:18:49.996824 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-72dffdd6-86aa-45dd-9817-27083da1265f req-781a8e93-4a46-41ca-923e-c2024f640dbb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c120d794-5327-4846-9553-2b508248915e", "name": "network-vif-deleted", "tag": "67b2df5a-1169-4fd1-bce0-0716babb1144"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:18:50.005632 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-72dffdd6-86aa-45dd-9817-27083da1265f req-781a8e93-4a46-41ca-923e-c2024f640dbb service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:50.006192 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-72dffdd6-86aa-45dd-9817-27083da1265f req-781a8e93-4a46-41ca-923e-c2024f640dbb service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:50.018598 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-72dffdd6-86aa-45dd-9817-27083da1265f req-781a8e93-4a46-41ca-923e-c2024f640dbb service nova] Creating event network-vif-deleted:67b2df5a-1169-4fd1-bce0-0716babb1144 for instance c120d794-5327-4846-9553-2b508248915e on hv-cinder-maste May 23 13:18:50.024436 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-72dffdd6-86aa-45dd-9817-27083da1265f req-781a8e93-4a46-41ca-923e-c2024f640dbb service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031279 May 23 13:18:50.025536 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 613/1232] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:18:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:18:50.587041 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:50.587724 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:50.594611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:50.595132 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:50.595891 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:50.596319 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:50.629314 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:50.629763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:50.630249 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lazy-loading 'fault' on Instance uuid c120d794-5327-4846-9553-2b508248915e {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:18:50.673817 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe5a3e6e6a0b4509b92cdcd54da0f4fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdfcfe4dbb42d3bb07585d85bff52a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:18:50.681651 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-315d85ef-8301-465f-a368-fd5cf0ed7ead tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 200 len: 1326 microversion: 2.1 time: 0.099005 May 23 13:18:50.682251 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 620/1233] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:50 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 1326 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:18:51.700479 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:51.701302 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:51.708025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:51.708474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:51.709241 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:51.709682 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:51.724645 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] HTTP exception thrown: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Request method failure captured: May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e HTTP/1.1 May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4hF4ZyDPnlDN9ASmMY792zN0OYK6Y1drU2nFqk0NVzaWI5z2I9j7lkgoigSVKq9VosdFCoar02lG9b3AEhGjB9W1bCcI_m3JMjSjqsUpbglkpd2vBa-iV0oVGhFl4I_pTRq_hcw0iv0Er_uhrZVRzFvswsLirgykyQWw81iHiO7-vc May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:18:51.727501 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: fe5a3e6e6a0b4509b92cdcd54da0f4fe May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:51.729316 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'c120d794-5327-4846-9553-2b508248915e'} May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:18:51.730596 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:51.732165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:51.733452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Returning 404 to user: Instance c120d794-5327-4846-9553-2b508248915e could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:18:51.733452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-d5844137-7e58-41fa-84be-34e1bdaaaf84 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 404 len: 111 microversion: 2.1 time: 0.032789 May 23 13:18:51.733452 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 614/1234] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:51 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:18:51.737822 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d59cb2d0-1351-4614-b958-4dacf1832561 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '.version_select at 0x7f9b12a3d160>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:51.858332 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d59cb2d0-1351-4614-b958-4dacf1832561 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-259479803" status: 202 len: 0 microversion: 2.1 time: 0.124014 May 23 13:18:51.859364 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 621/1235] 10.222.0.30 () {58 vars in 1217 bytes} [Mon May 23 13:18:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-259479803 => generated 0 bytes in 126 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:18:55.179638 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:55.180193 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:55.186681 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:55.187121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:55.187874 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:55.188309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:55.202064 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] HTTP exception thrown: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Request method failure captured: May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e HTTP/1.1 May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4hF4ZyDPnlDN9ASmMY792zN0OYK6Y1drU2nFqk0NVzaWI5z2I9j7lkgoigSVKq9VosdFCoar02lG9b3AEhGjB9W1bCcI_m3JMjSjqsUpbglkpd2vBa-iV0oVGhFl4I_pTRq_hcw0iv0Er_uhrZVRzFvswsLirgykyQWw81iHiO7-vc May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:18:55.203588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: fe5a3e6e6a0b4509b92cdcd54da0f4fe May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.205519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'c120d794-5327-4846-9553-2b508248915e'} May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:18:55.206483 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.208162 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:55.209926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Returning 404 to user: Instance c120d794-5327-4846-9553-2b508248915e could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:18:55.209926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-330cb2d2-e092-403c-84c9-10bd437deb4e tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 404 len: 111 microversion: 2.1 time: 0.029550 May 23 13:18:55.209926 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 615/1236] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:18:55 2022] DELETE /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:18:55.213767 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:55.214409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: c120d794-5327-4846-9553-2b508248915e] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:55.220939 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:55.221386 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:55.222180 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:55.222615 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:55.235838 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] HTTP exception thrown: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Request method failure captured: May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e HTTP/1.1 May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4hF4ZyDPnlDN9ASmMY792zN0OYK6Y1drU2nFqk0NVzaWI5z2I9j7lkgoigSVKq9VosdFCoar02lG9b3AEhGjB9W1bCcI_m3JMjSjqsUpbglkpd2vBa-iV0oVGhFl4I_pTRq_hcw0iv0Er_uhrZVRzFvswsLirgykyQWw81iHiO7-vc May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:18:55.237451 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: fe5a3e6e6a0b4509b92cdcd54da0f4fe May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.239172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'c120d794-5327-4846-9553-2b508248915e'} May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:18:55.240233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c120d794-5327-4846-9553-2b508248915e could not be found. May 23 13:18:55.242951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:55.244466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Returning 404 to user: Instance c120d794-5327-4846-9553-2b508248915e could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:18:55.244466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-f741c9d5-1079-41e9-8560-396cbee67bb6 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e" status: 404 len: 111 microversion: 2.1 time: 0.029548 May 23 13:18:55.244466 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 622/1237] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:55 2022] GET /compute/v2.1/servers/c120d794-5327-4846-9553-2b508248915e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:18:56.515430 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:56.515940 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:56.522522 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:56.523004 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:56.523754 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:56.524190 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:56.537114 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] HTTP exception thrown: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Request method failure captured: May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 HTTP/1.1 May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4hF4ZyDPnlDN9ASmMY792zN0OYK6Y1drU2nFqk0NVzaWI5z2I9j7lkgoigSVKq9VosdFCoar02lG9b3AEhGjB9W1bCcI_m3JMjSjqsUpbglkpd2vBa-iV0oVGhFl4I_pTRq_hcw0iv0Er_uhrZVRzFvswsLirgykyQWw81iHiO7-vc May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:18:56.538689 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: fe5a3e6e6a0b4509b92cdcd54da0f4fe May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.540523 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'a123fe36-b018-410e-a6a1-13c25f9713c1'} May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:18:56.541633 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.543502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:18:56.545242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Returning 404 to user: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:18:56.545242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-13382f65-383b-4001-a952-b7134128c154 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 404 len: 111 microversion: 2.1 time: 0.028416 May 23 13:18:56.545242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 616/1238] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:18:56 2022] DELETE /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:18:56.548178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:18:56.548783 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] [instance: a123fe36-b018-410e-a6a1-13c25f9713c1] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:18:56.555810 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:56.556246 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:56.556993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:18:56.557421 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:18:56.570652 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] HTTP exception thrown: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Request method failure captured: May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 HTTP/1.1 May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4hF4ZyDPnlDN9ASmMY792zN0OYK6Y1drU2nFqk0NVzaWI5z2I9j7lkgoigSVKq9VosdFCoar02lG9b3AEhGjB9W1bCcI_m3JMjSjqsUpbglkpd2vBa-iV0oVGhFl4I_pTRq_hcw0iv0Er_uhrZVRzFvswsLirgykyQWw81iHiO7-vc May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:18:56.572173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/11bdfcfe4dbb42d3bb07585d85bff52a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: 11bdfcfe4dbb42d3bb07585d85bff52a May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2097949922 May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: fe5a3e6e6a0b4509b92cdcd54da0f4fe May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-AttachVolumeTestJSON-2097949922-project May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: > May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.574053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'a123fe36-b018-410e-a6a1-13c25f9713c1'} May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:18:56.575179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. May 23 13:18:56.576775 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:18:56.578409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] Returning 404 to user: Instance a123fe36-b018-410e-a6a1-13c25f9713c1 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:18:56.578409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-365d634b-26be-401c-b7ef-f47598e44f55 tempest-AttachVolumeTestJSON-2097949922 tempest-AttachVolumeTestJSON-2097949922-project] 10.222.0.30 "GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1" status: 404 len: 111 microversion: 2.1 time: 0.029096 May 23 13:18:56.578409 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 623/1239] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:18:56 2022] GET /compute/v2.1/servers/a123fe36-b018-410e-a6a1-13c25f9713c1 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:19:01.875180 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-605b9757-c3a2-41d8-80ed-17c15f9cca6a tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Action: 'create', calling method: .version_select at 0x7f9b12f28670>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-2077046130", "metadata": {"Type": "work"}}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-605b9757-c3a2-41d8-80ed-17c15f9cca6a tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Request method failure captured: May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 122 May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4nFuQ91klLTvCIvHzmP8K3tMiRH-JNCFRih_CmzbGEjN4YoPx0r3BkZ-Anb2RuccEBmusk9TCQhSUzyLZc2ZBW9GJx6prt02pQUwTRrJWgOJyjAPZKnNzi9_hsPJp1uBBVcgt3ezf4WNviSqxyYiMrmLPpfdB5mGKgVv2tSxJaSb9E May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:19:01.885255 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 31e82249616a42c0973dcf3c0c065cd9 May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-2077046130", "metadata": {"Type": "work"}}} May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: May 23 13:19:01.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-2077046130', 'metadata': {'Type': 'work'}}}} May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise error May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi On instance['volume']: May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi '#$%' May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:19:01.888206 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 23 13:19:01.890141 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 23 13:19:01.891588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 23 13:19:01.891588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.891588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-605b9757-c3a2-41d8-80ed-17c15f9cca6a tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:19:01.891588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-605b9757-c3a2-41d8-80ed-17c15f9cca6a tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.125524 May 23 13:19:01.891588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 617/1240] 10.222.0.30 () {60 vars in 1155 bytes} [Mon May 23 13:19:01 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 127 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 13:19:01.897950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b01d5cbd-74bf-49f9-8c4f-f8ecb4efce8d tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Action: 'create', calling method: .version_select at 0x7f9b184868b0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1588196229", "metadata": {"Type": "work"}}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b01d5cbd-74bf-49f9-8c4f-f8ecb4efce8d tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Request method failure captured: May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 120 May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4nFuQ91klLTvCIvHzmP8K3tMiRH-JNCFRih_CmzbGEjN4YoPx0r3BkZ-Anb2RuccEBmusk9TCQhSUzyLZc2ZBW9GJx6prt02pQUwTRrJWgOJyjAPZKnNzi9_hsPJp1uBBVcgt3ezf4WNviSqxyYiMrmLPpfdB5mGKgVv2tSxJaSb9E May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:19:01.902178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 31e82249616a42c0973dcf3c0c065cd9 May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1588196229", "metadata": {"Type": "work"}}} May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: May 23 13:19:01.903665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1588196229', 'metadata': {'Type': 'work'}}}} May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise error May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi On instance['volume']: May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi '0' May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:19:01.904665 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 23 13:19:01.906442 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 23 13:19:01.907799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 23 13:19:01.907799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.907799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-b01d5cbd-74bf-49f9-8c4f-f8ecb4efce8d tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:19:01.907799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-b01d5cbd-74bf-49f9-8c4f-f8ecb4efce8d tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.009617 May 23 13:19:01.907799 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 624/1241] 10.222.0.30 () {60 vars in 1155 bytes} [Mon May 23 13:19:01 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 13:19:01.914117 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4fe951-bc86-484d-99fe-45ff74cef8d7 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Action: 'create', calling method: .version_select at 0x7f9b12da2940>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-298432543", "metadata": {"Type": "work"}}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4fe951-bc86-484d-99fe-45ff74cef8d7 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Request method failure captured: May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 118 May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4nFuQ91klLTvCIvHzmP8K3tMiRH-JNCFRih_CmzbGEjN4YoPx0r3BkZ-Anb2RuccEBmusk9TCQhSUzyLZc2ZBW9GJx6prt02pQUwTRrJWgOJyjAPZKnNzi9_hsPJp1uBBVcgt3ezf4WNviSqxyYiMrmLPpfdB5mGKgVv2tSxJaSb9E May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:19:01.918062 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 31e82249616a42c0973dcf3c0c065cd9 May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-298432543", "metadata": {"Type": "work"}}} May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: May 23 13:19:01.924669 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-298432543', 'metadata': {'Type': 'work'}}}} May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise error May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi On instance['volume']: May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi '' May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:19:01.925729 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 23 13:19:01.927584 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 23 13:19:01.929074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 23 13:19:01.929074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:01.929074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ea4fe951-bc86-484d-99fe-45ff74cef8d7 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:19:01.929074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ea4fe951-bc86-484d-99fe-45ff74cef8d7 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009151 May 23 13:19:01.929074 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 618/1242] 10.222.0.30 () {60 vars in 1155 bytes} [Mon May 23 13:19:01 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 13:19:01.932583 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9e11ae5a-3a26-46b1-91ca-5128e1110ecd tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Calling method '.version_select at 0x7f9b12a20310>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:19:01.961826 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-9e11ae5a-3a26-46b1-91ca-5128e1110ecd tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] HTTP exception thrown: Volume tempest-invalid-1029140328 could not be found. May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9e11ae5a-3a26-46b1-91ca-5128e1110ecd tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Request method failure captured: May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1029140328 HTTP/1.1 May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4nFuQ91klLTvCIvHzmP8K3tMiRH-JNCFRih_CmzbGEjN4YoPx0r3BkZ-Anb2RuccEBmusk9TCQhSUzyLZc2ZBW9GJx6prt02pQUwTRrJWgOJyjAPZKnNzi9_hsPJp1uBBVcgt3ezf4WNviSqxyYiMrmLPpfdB5mGKgVv2tSxJaSb9E May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:19:01.964449 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 31e82249616a42c0973dcf3c0c065cd9 May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Volume tempest-invalid-1029140328 could not be found. May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': 'tempest-invalid-1029140328'} May 23 13:19:01.966294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1029140328 could not be found. (HTTP 404) (Request-ID: req-8cb66a5f-d8d9-496e-8564-fe534249cc09) May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.967560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 13:19:01.969086 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1029140328 could not be found. May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:19:01.970815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1029140328 could not be found. May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1029140328 could not be found. May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9e11ae5a-3a26-46b1-91ca-5128e1110ecd tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Returning 404 to user: Volume tempest-invalid-1029140328 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9e11ae5a-3a26-46b1-91ca-5128e1110ecd tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1029140328" status: 404 len: 99 microversion: 2.1 time: 0.038455 May 23 13:19:01.972743 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 625/1243] 10.222.0.30 () {58 vars in 1217 bytes} [Mon May 23 13:19:01 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1029140328 => generated 99 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) May 23 13:19:01.976528 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3a4a01fd-0f22-43db-b9c1-1ad2f9177838 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004280 May 23 13:19:01.977025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 619/1244] 10.222.0.30 () {58 vars in 1139 bytes} [Mon May 23 13:19:01 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 23 13:19:01.988725 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-8ca87ad4-0017-4412-831b-698e4196074d tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003575 May 23 13:19:01.989219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 626/1245] 10.222.0.30 () {58 vars in 1136 bytes} [Mon May 23 13:19:01 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 23 13:19:01.998130 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6f84fe23-91cf-449e-9e0a-cdb6ebce199f tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Calling method '.version_select at 0x7f9b180a9a60>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-6f84fe23-91cf-449e-9e0a-cdb6ebce199f tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] HTTP exception thrown: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6f84fe23-91cf-449e-9e0a-cdb6ebce199f tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Request method failure captured: May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: DELETE /compute/v2.1/os-volumes/3552d949-7d22-44d7-a266-4e6f46f17526 HTTP/1.1 May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4nFuQ91klLTvCIvHzmP8K3tMiRH-JNCFRih_CmzbGEjN4YoPx0r3BkZ-Anb2RuccEBmusk9TCQhSUzyLZc2ZBW9GJx6prt02pQUwTRrJWgOJyjAPZKnNzi9_hsPJp1uBBVcgt3ezf4WNviSqxyYiMrmLPpfdB5mGKgVv2tSxJaSb9E May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:19:02.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-VolumesNegativeTest-2010023869 May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: 31e82249616a42c0973dcf3c0c065cd9 May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': '3552d949-7d22-44d7-a266-4e6f46f17526'} May 23 13:19:02.035163 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. (HTTP 404) (Request-ID: req-2362d0cb-6c46-4c4e-8fa8-094d1f3e3e9d) May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:02.036250 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 13:19:02.037859 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 23 13:19:02.039422 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-6f84fe23-91cf-449e-9e0a-cdb6ebce199f tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Returning 404 to user: Volume 3552d949-7d22-44d7-a266-4e6f46f17526 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-6f84fe23-91cf-449e-9e0a-cdb6ebce199f tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "DELETE /compute/v2.1/os-volumes/3552d949-7d22-44d7-a266-4e6f46f17526" status: 404 len: 109 microversion: 2.1 time: 0.037910 May 23 13:19:02.041659 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 620/1246] 10.222.0.30 () {58 vars in 1247 bytes} [Mon May 23 13:19:01 2022] DELETE /compute/v2.1/os-volumes/3552d949-7d22-44d7-a266-4e6f46f17526 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:19:02.043135 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cfda078f-733d-456d-9f60-170474d58c84 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Calling method '.version_select at 0x7f9b12a53ee0>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:19:02.070285 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.wsgi [None req-cfda078f-733d-456d-9f60-170474d58c84 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] HTTP exception thrown: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cfda078f-733d-456d-9f60-170474d58c84 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Request method failure captured: May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: request: GET /compute/v2.1/os-volumes/2135a9b1-e572-4594-919d-1bf8d330ae25 HTTP/1.1 May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept: application/json May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Accept-Encoding: identity May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Connection: close May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Length: 0 May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Content-Type: application/json May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Host: 10.222.0.30 May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Openstack-System-Scope: None May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: User-Agent: python-urllib3/1.26.9 May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Auth-Token: gAAAAABii4nFuQ91klLTvCIvHzmP8K3tMiRH-JNCFRih_CmzbGEjN4YoPx0r3BkZ-Anb2RuccEBmusk9TCQhSUzyLZc2ZBW9GJx6prt02pQUwTRrJWgOJyjAPZKnNzi9_hsPJp1uBBVcgt3ezf4WNviSqxyYiMrmLPpfdB5mGKgVv2tSxJaSb9E May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Id: None May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Domain-Name: None May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Identity-Status: Confirmed May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Is-Admin-Project: True May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Id: default May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Domain-Name: Default May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Project-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Role: member,reader May 23 13:19:02.073007 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Roles: member,reader May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/e9089605f3dc40408ce2fa54cf02fa9a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant: tempest-VolumesNegativeTest-2010023869 May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Id: e9089605f3dc40408ce2fa54cf02fa9a May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-Tenant-Name: tempest-VolumesNegativeTest-2010023869 May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Id: default May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Domain-Name: Default May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Id: 31e82249616a42c0973dcf3c0c065cd9 May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: X-User-Name: tempest-VolumesNegativeTest-2010023869-project May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: method: May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: exception: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: action_args: {'id': '2135a9b1-e572-4594-919d-1bf8d330ae25'} May 23 13:19:02.074796 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. (HTTP 404) (Request-ID: req-5f1c483b-ff0e-4598-87f4-c8724fa33809) May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:02.075782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 13:19:02.077097 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 13:19:02.079380 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR nova.api.openstack.wsgi May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cfda078f-733d-456d-9f60-170474d58c84 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] Returning 404 to user: Volume 2135a9b1-e572-4594-919d-1bf8d330ae25 could not be found. {{(pid=130045) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cfda078f-733d-456d-9f60-170474d58c84 tempest-VolumesNegativeTest-2010023869 tempest-VolumesNegativeTest-2010023869-project] 10.222.0.30 "GET /compute/v2.1/os-volumes/2135a9b1-e572-4594-919d-1bf8d330ae25" status: 404 len: 109 microversion: 2.1 time: 0.035581 May 23 13:19:02.080861 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 627/1247] 10.222.0.30 () {58 vars in 1244 bytes} [Mon May 23 13:19:02 2022] GET /compute/v2.1/os-volumes/2135a9b1-e572-4594-919d-1bf8d330ae25 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:19:43.092165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:19:43.092918 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:19:43.270986 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ff983ee3-12b1-416d-9f65-8c06d2d9b16c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Action: 'create', calling method: .version_select at 0x7f9b12e3a0d0>, body: {"keypair": {"name": "tempest-TestShelveInstance-469003220", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCL9Yk0bR9ZLCL1WepESKupOa4hb8Z1mrh2kGcGJdTXOTPv/MH/WfSyJppzb1brNVd/1Y7RqKR3uHXCnAmC2i/eu0tvu5CLzcbQ3VpvA92SkY5bPA/ToWHuhB9amJHdqCA=="}} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:19:43.286574 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-ff983ee3-12b1-416d-9f65-8c06d2d9b16c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Getting quotas for project c39739f8806b49638e73137c2403bae6. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:19:43.290519 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.quota [None req-ff983ee3-12b1-416d-9f65-8c06d2d9b16c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Getting quotas for user cf9dcdf4eb1643dc94c5d0e32f933205 and project c39739f8806b49638e73137c2403bae6. Resources: {'key_pairs'} {{(pid=130046) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:19:43.303588 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR oslo.messaging._drivers.impl_rabbit [None req-ff983ee3-12b1-416d-9f65-8c06d2d9b16c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [819dbb27-e0ee-4967-9b27-a1c2e4337b56] AMQP server on 10.222.0.30:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 13:19:44.310811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [None req-ff983ee3-12b1-416d-9f65-8c06d2d9b16c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [819dbb27-e0ee-4967-9b27-a1c2e4337b56] Reconnected to AMQP server on 10.222.0.30:5672 via [amqp] client with port 46486. May 23 13:19:44.377372 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ff983ee3-12b1-416d-9f65-8c06d2d9b16c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 1.286646 May 23 13:19:44.378000 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 621/1248] 10.222.0.30 () {60 vars in 1158 bytes} [Mon May 23 13:19:43 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 1288 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:19:59.715087 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:19:59.715592 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:19:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:20:07.313099 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=23,nova_api:DELETE=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:07.313843 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=54,nova_cell1:INSERT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:07.318660 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1271075147", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-469003220", "security_groups": [{"name": "tempest-secgroup-smoke-1057148063"}], "networks": [{"uuid": "9631a516-9a27-49d0-91a6-c381eb1d1c3c"}], "block_device_mapping_v2": [{"uuid": "6aea8b0d-702f-4926-9957-b932698402e4", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:20:07.428217 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.network.neutron [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] validate_networks() for [('9631a516-9a27-49d0-91a6-c381eb1d1c3c', None, None, None, None, None)] {{(pid=130045) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 13:20:08.129563 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Flavor limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 13:20:08.130043 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Image limits 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 13:20:08.130526 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Flavor pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 13:20:08.130927 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Image pref 0:0:0 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 13:20:08.131331 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.virt.hardware [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130045) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 13:20:08.139918 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:08.140360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:08.141150 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:08.141836 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:08.160225 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Getting quotas for project c39739f8806b49638e73137c2403bae6. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 13:20:08.163988 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.quota [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Getting quotas for user cf9dcdf4eb1643dc94c5d0e32f933205 and project c39739f8806b49638e73137c2403bae6. Resources: {'instances', 'ram', 'cores'} {{(pid=130045) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 13:20:08.219611 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Going to run 1 instances... {{(pid=130045) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 13:20:08.222388 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='6aea8b0d-702f-4926-9957-b932698402e4',volume_size=None,volume_type=None)] {{(pid=130045) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 13:20:08.587841 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d2c80e76-c36b-491a-b85f-042d9eebeed1 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.277231 May 23 13:20:08.588795 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 628/1249] 10.222.0.30 () {60 vars in 1146 bytes} [Mon May 23 13:20:07 2022] POST /compute/v2.1/servers => generated 406 bytes in 1279 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 13:20:08.595941 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=4 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:08.597586 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:08.601520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-57a38d48-1113-416f-bf5e-50bd2d6a7a6e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:08.602203 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-57a38d48-1113-416f-bf5e-50bd2d6a7a6e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:08.658392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-57a38d48-1113-416f-bf5e-50bd2d6a7a6e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:08.666612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-57a38d48-1113-416f-bf5e-50bd2d6a7a6e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1088 microversion: 2.1 time: 0.071944 May 23 13:20:08.667178 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 622/1250] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:08 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1088 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:09.685014 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:09.685610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:09.692477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:09.692919 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:09.693676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:09.694133 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:09.758952 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:09.767004 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-bad465a8-de4b-4c33-886a-4a6dbaa51eb4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1138 microversion: 2.1 time: 0.086348 May 23 13:20:09.767570 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 629/1251] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:09 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1138 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:10.784282 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:10.784753 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:10.793151 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:10.793764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:10.794566 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:10.795291 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:10.798370 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:20:10.801030 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:20:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 (ip 10.222.0.30) !!! May 23 13:20:10.892310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:10.900143 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5a3df617-17a9-4f06-8cb3-141c4660e94c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1204 microversion: 2.1 time: 0.119721 May 23 13:20:10.900743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 623/1252] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:10 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1204 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:11.917406 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:11.918041 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:11.924416 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:11.924863 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:11.926880 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:11.927539 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:12.159163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:12.168310 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d8811dd5-279b-4c79-b2e6-c2825517045a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.254509 May 23 13:20:12.168983 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 630/1253] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:11 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:12.364782 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-48c41a7b-c4b3-47e2-9767-0ce25ed80b3b req-1e95e3e9-f6d5-48e7-bb2a-335a84c64a46 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:20:12.374154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-48c41a7b-c4b3-47e2-9767-0ce25ed80b3b req-1e95e3e9-f6d5-48e7-bb2a-335a84c64a46 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:12.374154 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-48c41a7b-c4b3-47e2-9767-0ce25ed80b3b req-1e95e3e9-f6d5-48e7-bb2a-335a84c64a46 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:12.395502 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-48c41a7b-c4b3-47e2-9767-0ce25ed80b3b req-1e95e3e9-f6d5-48e7-bb2a-335a84c64a46 service nova] Creating event network-changed:2bd9e090-be1e-45db-8933-3657fc0a924e for instance fb07f17d-e467-4c67-aed5-3a14ad46d804 on hv-cinder-maste May 23 13:20:12.401119 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-48c41a7b-c4b3-47e2-9767-0ce25ed80b3b req-1e95e3e9-f6d5-48e7-bb2a-335a84c64a46 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042188 May 23 13:20:12.401737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 624/1254] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:20:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:20:13.187583 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:13.188197 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:13.195508 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:13.195953 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:13.196713 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:13.197147 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:13.326945 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:13.335806 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-c5ffad88-9c6d-4ddc-9b8e-6c3e83e9d3eb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.151742 May 23 13:20:13.336631 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 631/1255] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:13 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:14.359804 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:14.360505 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:14.366852 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:14.367309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:14.368079 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:14.368517 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:14.507717 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:14.515158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7783bde6-9700-4bd9-bd24-fcb795c5d674 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.160074 May 23 13:20:14.515732 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 625/1256] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:14 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:20:15.532151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:15.532767 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:15.539220 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:15.539676 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:15.540440 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:15.540879 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:15.671173 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:15.678781 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6539d21a-964c-4734-85d6-4d5095cf44ee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.150459 May 23 13:20:15.683072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 632/1257] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:15 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:16.696209 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:16.696826 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:16.703442 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:16.703889 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:16.704647 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:16.705081 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:16.837615 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:16.845741 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-5d8e6e8a-87c5-4f5c-a2ed-eebae7dfff29 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.154102 May 23 13:20:16.846322 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 626/1258] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:16 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:17.863360 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:17.864127 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:17.871243 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:17.871685 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:17.872434 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:17.872863 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:18.006596 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:18.014074 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-e0af1e20-151f-45d0-85b1-c6f15adf47ed tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.154477 May 23 13:20:18.014669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 633/1259] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:17 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:19.031847 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:19.032524 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:19.035018 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell0:SELECT=2 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:19.040029 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:19.040472 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:19.041521 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:19.041977 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:19.175539 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:19.183325 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cc654aa2-303f-4e01-80f8-24712e915304 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.155432 May 23 13:20:19.183990 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 627/1260] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:19 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:20.201151 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:20.201820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:20.203749 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:20.204399 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell0:SELECT=6 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:20:20.210560 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:20.210914 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:20.211678 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:20.212138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:20.346837 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:20.355061 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fcacd668-3ba0-461f-a894-d849e8a3764f tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.157872 May 23 13:20:20.355618 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 634/1261] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:20 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:21.372480 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:21.373083 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:21.379561 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:21.379998 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:21.380743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:21.381806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:21.509564 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:21.517311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-ff7147d3-5447-4a5b-8082-14886939a7b9 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.148692 May 23 13:20:21.517939 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 628/1262] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:21 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:21.907666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-fbed0ee0-3ce1-4454-8567-c4069b591908 req-7f0e0836-7871-4f1b-b02e-669b6b4bcf5f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "name": "network-vif-plugged", "status": "completed", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:20:21.917198 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-fbed0ee0-3ce1-4454-8567-c4069b591908 req-7f0e0836-7871-4f1b-b02e-669b6b4bcf5f service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:21.917638 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-fbed0ee0-3ce1-4454-8567-c4069b591908 req-7f0e0836-7871-4f1b-b02e-669b6b4bcf5f service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:21.928118 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-fbed0ee0-3ce1-4454-8567-c4069b591908 req-7f0e0836-7871-4f1b-b02e-669b6b4bcf5f service nova] Creating event network-vif-plugged:2bd9e090-be1e-45db-8933-3657fc0a924e for instance fb07f17d-e467-4c67-aed5-3a14ad46d804 on hv-cinder-maste May 23 13:20:21.933202 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-fbed0ee0-3ce1-4454-8567-c4069b591908 req-7f0e0836-7871-4f1b-b02e-669b6b4bcf5f service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029118 May 23 13:20:21.933759 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 635/1263] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:20:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:20:22.534188 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:22.534740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:22.541943 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:22.542391 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:22.543152 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:22.543591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:22.672381 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:22.680455 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-c361f882-b00a-42ff-9c5e-7ad1a748b723 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.150180 May 23 13:20:22.681026 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 629/1264] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:22 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:23.699176 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:23.699961 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:23.706487 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:23.706933 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:23.707695 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:23.708128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:23.839533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:23.846995 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-9c105e62-a4d1-4d99-9a11-1b0fd97ec72b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1260 microversion: 2.1 time: 0.153330 May 23 13:20:23.847548 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 636/1265] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:23 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:24.864048 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:24.864774 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:24.871363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:24.871811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:24.872612 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:24.873084 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:25.009456 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:25.017334 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-df10baa0-e821-4fbf-93f1-77898716e0f7 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1443 microversion: 2.1 time: 0.157364 May 23 13:20:25.021511 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 630/1266] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:24 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1443 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:25.033209 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:20:25.033815 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:20:25.040480 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:25.040924 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:25.041681 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:25.042153 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:25.170993 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:20:25.178972 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d4291f98-88bf-4cb0-a9cb-83e0a7c536e0 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1443 microversion: 2.1 time: 0.149513 May 23 13:20:25.179538 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 637/1267] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:20:25 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1443 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:20:27.406526 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-4bd275ed-f2b9-4f9b-be76-5cdfc5864d2b req-e552279d-d9d5-488a-b592-415e9cfab942 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:20:27.415071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4bd275ed-f2b9-4f9b-be76-5cdfc5864d2b req-e552279d-d9d5-488a-b592-415e9cfab942 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:20:27.415520 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-4bd275ed-f2b9-4f9b-be76-5cdfc5864d2b req-e552279d-d9d5-488a-b592-415e9cfab942 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:20:27.426269 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-4bd275ed-f2b9-4f9b-be76-5cdfc5864d2b req-e552279d-d9d5-488a-b592-415e9cfab942 service nova] Creating event network-changed:2bd9e090-be1e-45db-8933-3657fc0a924e for instance fb07f17d-e467-4c67-aed5-3a14ad46d804 on hv-cinder-maste May 23 13:20:27.431572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-4bd275ed-f2b9-4f9b-be76-5cdfc5864d2b req-e552279d-d9d5-488a-b592-415e9cfab942 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087098 May 23 13:20:27.432156 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 631/1268] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:20:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:21:39.318172 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:21:39.318780 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:21:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:22:41.243591 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:41.244338 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=52 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:41.248820 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:22:41.249344 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:41.255138 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:41.259053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:41.259053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:41.259053 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:41.280072 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lazy-loading 'flavor' on Instance uuid fb07f17d-e467-4c67-aed5-3a14ad46d804 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:22:41.545533 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-d5f76fd9-2994-4b39-94f7-dcf4860fb5f5 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "POST /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804/action" status: 202 len: 0 microversion: 2.1 time: 0.303283 May 23 13:22:41.546161 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 638/1269] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:22:41 2022] POST /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804/action => generated 0 bytes in 305 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:22:41.552875 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR oslo.messaging._drivers.impl_rabbit [-] [e3f86a41-e5e3-44ca-8874-4d9c252761c7] AMQP server on 10.222.0.30:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 23 13:22:41.553491 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=22 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:41.554274 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=46 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:41.558412 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:41.559123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:41.565268 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:41.565720 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:41.566527 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:41.566968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:41.812935 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:41.820392 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0b8de89b-1316-4900-abc6-807a4ad4eda6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.269778 May 23 13:22:41.820951 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 632/1270] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:41 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:42.837666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:42.838401 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:42.845311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:42.845764 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:42.846564 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:42.847005 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:43.055778 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:43.063358 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7e7f9c70-c174-43a9-aabc-22820639428e tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.229948 May 23 13:22:43.063911 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 639/1271] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:42 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:44.080505 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:44.081075 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:44.088600 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:44.089045 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:44.089811 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:44.090857 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:44.097049 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] [e3f86a41-e5e3-44ca-8874-4d9c252761c7] Reconnected to AMQP server on 10.222.0.30:5672 via [amqp] client with port 46736. May 23 13:22:44.220765 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:44.228120 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-7a17763c-c61a-4544-946d-06dcd2be7852 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.151674 May 23 13:22:44.228684 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 633/1272] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:44 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:45.245128 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:45.245722 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:45.252207 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:45.252649 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:45.253396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:45.253850 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:45.384444 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:45.392376 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-498cf613-fa57-48c2-94b0-23e45a841465 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.151077 May 23 13:22:45.392937 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 640/1273] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:45 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:46.409076 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:46.409691 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:46.416123 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:46.416554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:46.417309 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:46.417743 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:46.549008 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:46.556407 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a9c01db7-319c-4537-add4-d093a8a89bef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.151232 May 23 13:22:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 634/1274] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:46 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:47.574111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:47.575076 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:47.581403 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:47.581842 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:47.582707 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:47.583154 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:47.713770 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:47.721661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-03cf2d40-8431-46e1-af93-6b6b7f02745d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.151437 May 23 13:22:47.722255 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 641/1275] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:47 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:48.739496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:48.740112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:48.747005 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:48.747471 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:48.748242 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:48.748794 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:48.880999 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:48.888806 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-833a99ce-df13-4981-a58f-3b0e7f67c9d4 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.152900 May 23 13:22:48.889361 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 635/1276] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:48 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:49.905726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:49.906356 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:49.912719 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:49.913162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:49.913946 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:49.914378 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:50.043257 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:50.050569 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-fd265a5a-3b8a-4303-9aa6-2740528d83d8 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.148730 May 23 13:22:50.051124 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 642/1277] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:49 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:51.068199 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:51.068745 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:51.075740 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:51.076181 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:51.076936 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:51.077369 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:51.207845 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:51.215960 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3c58b395-9daf-474b-aef1-244ba4e66627 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.151623 May 23 13:22:51.216521 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 636/1278] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:51 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:52.232679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:52.233221 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:52.239776 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:52.240224 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:52.240981 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:52.241421 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:52.445199 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:52.453280 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-59684ccb-6b95-4b7e-a951-b4d28e1fb71d tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.225148 May 23 13:22:52.453856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 643/1279] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:52 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:53.470319 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:53.470925 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:53.477324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:53.477769 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:53.478554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:53.478985 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:53.614838 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:53.622283 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-313a63df-8e47-465e-892d-d7cb3bcc1727 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.155960 May 23 13:22:53.622840 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 637/1280] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:53 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:54.640348 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:54.646059 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:54.647464 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:54.647914 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:54.648666 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:54.649096 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:54.783630 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:54.791465 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ab2f3eb6-f6e0-4ea3-a8a3-da67514f3e06 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.156158 May 23 13:22:54.792033 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 644/1281] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:54 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:55.810267 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:55.810831 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:55.817382 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:55.817836 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:55.818653 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:55.819089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:55.949545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:55.957025 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-bca5ef66-e9ce-40e8-96ee-4b54a3f1c48c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1507 microversion: 2.1 time: 0.151393 May 23 13:22:55.957591 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 638/1282] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:55 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1507 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:55.973661 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:22:55.974263 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:55.981424 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:55.981871 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:55.982657 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:55.983103 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:56.157553 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-5fb55d58-a3e4-4a33-8078-85d7852fd551 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "POST /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804/action" status: 202 len: 0 microversion: 2.1 time: 0.187492 May 23 13:22:56.158220 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 645/1283] 10.222.0.30 () {60 vars in 1277 bytes} [Mon May 23 13:22:55 2022] POST /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804/action => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:22:56.166226 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:56.166837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:56.172901 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:56.174613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:56.174613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:56.174613 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:56.341121 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:56.348829 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-8c06063e-f7b5-4bcc-9c59-5fbe0ea67564 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1515 microversion: 2.1 time: 0.186501 May 23 13:22:56.355764 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 639/1284] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:56 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1515 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:56.485163 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-f9e2939e-31af-485d-bf56-db9b2f8831d6 req-119da9aa-ac9d-4b56-80c8-2fdad9acf8e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "name": "network-vif-unplugged", "status": "completed", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:22:56.493294 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-f9e2939e-31af-485d-bf56-db9b2f8831d6 req-119da9aa-ac9d-4b56-80c8-2fdad9acf8e3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:56.493738 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-f9e2939e-31af-485d-bf56-db9b2f8831d6 req-119da9aa-ac9d-4b56-80c8-2fdad9acf8e3 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:56.509529 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.compute.server_external_events [req-f9e2939e-31af-485d-bf56-db9b2f8831d6 req-119da9aa-ac9d-4b56-80c8-2fdad9acf8e3 service nova] Unable to find a host for instance fb07f17d-e467-4c67-aed5-3a14ad46d804. Dropping event network-vif-unplugged {{(pid=130045) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} May 23 13:22:56.510546 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-f9e2939e-31af-485d-bf56-db9b2f8831d6 req-119da9aa-ac9d-4b56-80c8-2fdad9acf8e3 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.095362 May 23 13:22:56.511134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 646/1285] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:22:56 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 97 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) May 23 13:22:57.367094 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:57.367727 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:57.375102 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:57.375545 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:57.376298 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:57.376768 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:57.506848 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:57.515267 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-2dcd8587-c722-4196-8153-401557437670 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1517 microversion: 2.1 time: 0.152006 May 23 13:22:57.515825 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 640/1286] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:57 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1517 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:58.532639 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:58.533311 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:58.539958 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:58.540415 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:58.541204 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:58.541637 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:58.688260 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:58.696573 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-4188e80b-f807-4c3c-8414-d33f3001e5c6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1517 microversion: 2.1 time: 0.167559 May 23 13:22:58.698493 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 647/1287] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:58 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1517 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:22:59.716617 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:22:59.717237 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:22:59.724052 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:59.724500 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:59.725486 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:22:59.725953 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:22:59.854755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:22:59.863071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-3fd65667-6f9e-4460-9496-96a01509d915 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.151170 May 23 13:22:59.863644 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 641/1288] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:22:59 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:00.880270 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:00.880994 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:00.888960 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:00.892999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:00.892999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:00.892999 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:01.269382 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:01.277160 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cd58f6f2-79c6-4fac-82c9-2ac7e4f1354b tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.400484 May 23 13:23:01.277742 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 648/1289] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:00 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:02.295044 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:02.295594 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:02.302057 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:02.302579 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:02.303337 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:02.303766 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:02.440112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:02.447962 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-fc0be712-876a-4f4a-9001-3d30573bb42a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.157785 May 23 13:23:02.448540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 642/1290] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:02 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:03.466264 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:03.466875 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:03.473430 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:03.473898 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:03.474672 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:03.475106 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:03.609335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:03.616679 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-a6b8d75a-b372-4687-ad23-ade801c0df38 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.155390 May 23 13:23:03.617233 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 649/1291] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:03 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:04.634220 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:04.634837 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:04.641789 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:04.642258 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:04.643008 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:04.643439 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:04.773337 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:04.781338 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-0406372c-d429-4150-b033-4ff70c222850 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.151080 May 23 13:23:04.781929 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 643/1292] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:04 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:05.802279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:05.802830 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:05.810219 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:05.810669 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:05.811551 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:05.812012 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:05.941013 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:05.948976 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-cfc906da-9dab-42e3-bb67-8c6ea2db6808 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.150103 May 23 13:23:05.949599 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 650/1293] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:05 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:06.966165 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:06.966885 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:06.974499 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:06.974944 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:06.975708 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:06.976138 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:07.107860 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:07.115442 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-87a06293-5a7d-4659-8c59-0763e568f135 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.153764 May 23 13:23:07.116009 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 644/1294] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:06 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:08.133335 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:08.133963 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:08.141279 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:08.141726 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:08.142514 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:08.142950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:08.271982 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:08.280541 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-6e70dc27-11a3-4522-8c07-ea71694a8cee tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.151074 May 23 13:23:08.281111 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 651/1295] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:08 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:09.297075 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:09.297671 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:09.304158 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:09.304605 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:09.305363 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:09.305800 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:09.435474 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:09.442830 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-cc447f20-d2d8-4d18-8bc3-c133789cb2ac tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.149574 May 23 13:23:09.443394 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 645/1296] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:09 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:10.459865 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:10.460477 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:10.466867 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:10.467320 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:10.468083 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:10.468520 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:10.472886 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:10.518217 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [req-95f8e838-3f6b-4f51-b0ed-63742ccd2fa9 req-4c9ae3fb-ac2e-421e-a617-c8c5ceab9cb4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "name": "network-vif-plugged", "status": "completed", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:23:10.527664 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-95f8e838-3f6b-4f51-b0ed-63742ccd2fa9 req-4c9ae3fb-ac2e-421e-a617-c8c5ceab9cb4 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:10.528115 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [req-95f8e838-3f6b-4f51-b0ed-63742ccd2fa9 req-4c9ae3fb-ac2e-421e-a617-c8c5ceab9cb4 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:10.539508 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.compute.server_external_events [req-95f8e838-3f6b-4f51-b0ed-63742ccd2fa9 req-4c9ae3fb-ac2e-421e-a617-c8c5ceab9cb4 service nova] Creating event network-vif-plugged:2bd9e090-be1e-45db-8933-3657fc0a924e for instance fb07f17d-e467-4c67-aed5-3a14ad46d804 on hv-cinder-maste May 23 13:23:10.545186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [req-95f8e838-3f6b-4f51-b0ed-63742ccd2fa9 req-4c9ae3fb-ac2e-421e-a617-c8c5ceab9cb4 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030593 May 23 13:23:10.545868 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 646/1297] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:23:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 13:23:10.611729 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:10.619602 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-1cde3d7b-2091-42a7-9956-c04357d8f43c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.163459 May 23 13:23:10.620856 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 652/1298] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:10 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:11.639635 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:11.640324 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:11.642236 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=31 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:11.647095 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:11.647540 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:11.648295 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:11.648724 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:11.785151 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:11.787323 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_cell1:SELECT=91 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:11.793787 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-822dcb85-9f36-4be4-a29e-97c4043c7f50 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1573 microversion: 2.1 time: 0.159794 May 23 13:23:11.794419 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 647/1299] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:11 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1573 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:12.810782 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:23:12.811396 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:23:12.814171 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=33 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:12.819003 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:12.825168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:12.825168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:23:12.825168 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:23:12.956805 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:23:12.965054 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-ffcda38d-dbc0-4525-b7fe-e46d90a52423 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1560 microversion: 2.1 time: 0.158189 May 23 13:23:12.965604 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 653/1300] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:23:12 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:23:30.826186 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:23:30.827126 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:23:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:24:42.861379 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG dbcounter [-] [130046] Writing DB stats nova_api:SELECT=1 {{(pid=130046) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:42.861995 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_api:SELECT=1 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:42.862777 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG dbcounter [-] [130045] Writing DB stats nova_cell1:SELECT=10 {{(pid=130045) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:42.866950 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-123f7c79-3bbe-4ff4-bed7-ea6ab235130f req-0d174acd-c1f4-47c8-bbe0-772bc2b2c9f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:24:42.874929 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-123f7c79-3bbe-4ff4-bed7-ea6ab235130f req-0d174acd-c1f4-47c8-bbe0-772bc2b2c9f2 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:42.875512 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-123f7c79-3bbe-4ff4-bed7-ea6ab235130f req-0d174acd-c1f4-47c8-bbe0-772bc2b2c9f2 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:42.887589 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-123f7c79-3bbe-4ff4-bed7-ea6ab235130f req-0d174acd-c1f4-47c8-bbe0-772bc2b2c9f2 service nova] Creating event network-changed:2bd9e090-be1e-45db-8933-3657fc0a924e for instance fb07f17d-e467-4c67-aed5-3a14ad46d804 on hv-cinder-maste May 23 13:24:42.892870 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-123f7c79-3bbe-4ff4-bed7-ea6ab235130f req-0d174acd-c1f4-47c8-bbe0-772bc2b2c9f2 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034001 May 23 13:24:42.893664 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 654/1301] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:24:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:24:42.917572 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:42.918107 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:24:42.925089 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:42.925813 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:42.926886 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:42.927518 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:42.959191 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Going to try to terminate instance {{(pid=130046) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 13:24:43.272365 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-dc1d2f65-002c-4906-b940-1385f01258de tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "DELETE /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 204 len: 0 microversion: 2.1 time: 0.416110 May 23 13:24:43.273258 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 648/1302] 10.222.0.30 () {58 vars in 1238 bytes} [Mon May 23 13:24:42 2022] DELETE /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 13:24:43.283757 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:43.284469 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:24:43.291178 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:43.291852 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:43.292613 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:43.293042 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:43.432712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:24:43.441558 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-7879936d-7bb3-4f9f-b58c-534fbe46e207 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.163909 May 23 13:24:43.442162 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 655/1303] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:24:43 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:24:44.459035 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:44.459755 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:24:44.465796 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:44.466271 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:44.467060 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:44.467595 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:44.609554 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:24:44.617022 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-60e48edc-66a1-4b17-addf-458b6b490e9c tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1566 microversion: 2.1 time: 0.162226 May 23 13:24:44.617576 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 649/1304] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:24:44 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1566 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:24:45.607869 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [req-0077250d-2572-45ee-bbb2-a93bb2909e6d req-65b53e13-de09-4c6a-895f-a28d916c8750 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fb07f17d-e467-4c67-aed5-3a14ad46d804", "name": "network-vif-deleted", "tag": "2bd9e090-be1e-45db-8933-3657fc0a924e"}]} {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 13:24:45.615692 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-0077250d-2572-45ee-bbb2-a93bb2909e6d req-65b53e13-de09-4c6a-895f-a28d916c8750 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:45.616134 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [req-0077250d-2572-45ee-bbb2-a93bb2909e6d req-65b53e13-de09-4c6a-895f-a28d916c8750 service nova] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:45.626854 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.compute.server_external_events [req-0077250d-2572-45ee-bbb2-a93bb2909e6d req-65b53e13-de09-4c6a-895f-a28d916c8750 service nova] Creating event network-vif-deleted:2bd9e090-be1e-45db-8933-3657fc0a924e for instance fb07f17d-e467-4c67-aed5-3a14ad46d804 on hv-cinder-maste May 23 13:24:45.631952 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [req-0077250d-2572-45ee-bbb2-a93bb2909e6d req-65b53e13-de09-4c6a-895f-a28d916c8750 service nova] 10.222.0.30 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082462 May 23 13:24:45.635483 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 656/1305] 10.222.0.30 () {64 vars in 1317 bytes} [Mon May 23 13:24:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 13:24:45.636023 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:45.636733 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:24:45.643868 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:45.644310 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:45.645060 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:45.645496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:45.707674 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.policy [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130046) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:24:45.715182 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-a5059d5d-1a4d-472c-9170-a536cd250e8a tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1381 microversion: 2.1 time: 0.083449 May 23 13:24:45.715737 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 650/1306] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:24:45 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1381 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 13:24:46.733763 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:46.734498 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.compute.api [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130045) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:24:46.740610 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:46.742985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:46.742985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:46.742985 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:46.773017 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:46.773462 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG oslo_concurrency.lockutils [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130045) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:46.773951 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.objects.instance [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lazy-loading 'fault' on Instance uuid fb07f17d-e467-4c67-aed5-3a14ad46d804 {{(pid=130045) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 13:24:46.819129 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.policy [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cf9dcdf4eb1643dc94c5d0e32f933205', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c39739f8806b49638e73137c2403bae6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130045) authorize /opt/stack/nova/nova/policy.py:199}} May 23 13:24:46.826464 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-47973652-d36a-43eb-9849-174a3c02ffef tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 200 len: 1224 microversion: 2.1 time: 0.097509 May 23 13:24:46.827688 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 657/1307] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:24:46 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 1224 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 13:24:47.849609 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '>' {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:47.850253 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.compute.api [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [instance: fb07f17d-e467-4c67-aed5-3a14ad46d804] Fetching instance by UUID {{(pid=130046) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 13:24:47.856865 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:47.857311 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:47.858112 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 13:24:47.858810 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG oslo_concurrency.lockutils [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Lock "3c6cf08e-95e3-44d9-8446-eef7c93b66c5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130046) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 13:24:47.871625 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.wsgi [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] HTTP exception thrown: Instance fb07f17d-e467-4c67-aed5-3a14ad46d804 could not be found. May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Request method failure captured: May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: request: GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 HTTP/1.1 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept: application/json May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Accept-Encoding: identity May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Connection: close May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Length: 0 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Content-Type: application/json May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Host: 10.222.0.30 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Openstack-System-Scope: None May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: User-Agent: python-urllib3/1.26.9 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Auth-Token: gAAAAABii4not8hqaRYqtGg9BnJayoQD0hzNi9ajptBQNhYvN95Ouoa5OU9hNIwdduNYXo4EoWwbsIlcq-a4qSDyQoB0m80CpuEIQKJJ1896Wou6_zFQFPlEQrLRMgWDQ0MA0tlOQq9xszf-5KkM80TtcXoAO8Eh4pbAytBdESXveNkRxFb_wV4 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Id: None May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Domain-Name: None May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Identity-Status: Confirmed May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Is-Admin-Project: True May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Id: default May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Domain-Name: Default May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Id: c39739f8806b49638e73137c2403bae6 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Project-Name: tempest-TestShelveInstance-1975399809 May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Role: member,reader May 23 13:24:47.873233 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Roles: member,reader May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.30:8080", "publicURL": "http://10.222.0.30:8080/v1/AUTH_c39739f8806b49638e73137c2403bae6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c39739f8806b49638e73137c2403bae6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2/c39739f8806b49638e73137c2403bae6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/volume/v3/c39739f8806b49638e73137c2403bae6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.30/image"}]}] May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant: tempest-TestShelveInstance-1975399809 May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Id: c39739f8806b49638e73137c2403bae6 May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-Tenant-Name: tempest-TestShelveInstance-1975399809 May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User: tempest-TestShelveInstance-1975399809-project May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Id: default May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Domain-Name: Default May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Id: cf9dcdf4eb1643dc94c5d0e32f933205 May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: X-User-Name: tempest-TestShelveInstance-1975399809-project May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: method: > May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: exception: Instance fb07f17d-e467-4c67-aed5-3a14ad46d804 could not be found. May 23 13:24:47.875496 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: action_args: {'id': 'fb07f17d-e467-4c67-aed5-3a14ad46d804'} May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: {{(pid=130046) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fb07f17d-e467-4c67-aed5-3a14ad46d804 could not be found. May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 13:24:47.876607 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fb07f17d-e467-4c67-aed5-3a14ad46d804 could not be found. May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fb07f17d-e467-4c67-aed5-3a14ad46d804 could not be found. May 23 13:24:47.878408 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: ERROR nova.api.openstack.wsgi May 23 13:24:47.879949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: DEBUG nova.api.openstack.wsgi [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Returning 404 to user: Instance fb07f17d-e467-4c67-aed5-3a14ad46d804 could not be found. {{(pid=130046) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 13:24:47.879949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO nova.api.openstack.requestlog [None req-54222595-657c-4b18-9eba-61179110faf6 tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804" status: 404 len: 111 microversion: 2.1 time: 0.028989 May 23 13:24:47.879949 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: [pid: 130046|app: 0|req: 651/1308] 10.222.0.30 () {58 vars in 1235 bytes} [Mon May 23 13:24:47 2022] GET /compute/v2.1/servers/fb07f17d-e467-4c67-aed5-3a14ad46d804 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 13:24:48.093512 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: DEBUG nova.api.openstack.wsgi [None req-56a1399b-d091-40f8-9529-d155eee2cafb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] Calling method '.version_select at 0x7f9b12a3bd30>' {{(pid=130045) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 13:24:48.096671 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: ERROR oslo.messaging._drivers.impl_rabbit [None req-56a1399b-d091-40f8-9529-d155eee2cafb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [91422227-95e9-4650-ad38-b478e8ff007f] AMQP server on 10.222.0.30:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 13:24:49.104144 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [None req-56a1399b-d091-40f8-9529-d155eee2cafb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] [91422227-95e9-4650-ad38-b478e8ff007f] Reconnected to AMQP server on 10.222.0.30:5672 via [amqp] client with port 46886. May 23 13:24:49.188036 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO nova.api.openstack.requestlog [None req-56a1399b-d091-40f8-9529-d155eee2cafb tempest-TestShelveInstance-1975399809 tempest-TestShelveInstance-1975399809-project] 10.222.0.30 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-469003220" status: 202 len: 0 microversion: 2.1 time: 1.098352 May 23 13:24:49.188620 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: [pid: 130045|app: 0|req: 658/1309] 10.222.0.30 () {58 vars in 1250 bytes} [Mon May 23 13:24:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-469003220 => generated 0 bytes in 1100 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 13:24:59.725633 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:24:59.726283 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:24:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:25:04.833071 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:25:04.833690 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:25:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:26:50.835946 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:26:50.836570 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:26:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:28:19.735712 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:28:19.736179 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:28:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:28:24.841286 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:28:24.841891 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:28:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:29:59.331812 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:29:59.332325 dv-cinder-master-3-iscsi devstack@n-api.service[130045]: Mon May 23 13:29:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:30:10.844092 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:30:10.844637 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:30:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 13:30:25.714492 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 13:30:25.715099 dv-cinder-master-3-iscsi devstack@n-api.service[130046]: Mon May 23 13:30:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!